You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2015/12/20 01:40:55 UTC

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1051 - Still Failing

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1051/

1 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=59568, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=59568, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
	at __randomizedtesting.SeedInfo.seed([624CA185227AE717:EA189E5F8C868AEF]:0)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:60188: Could not find collection : awholynewstresscollection_collection4_2
	at __randomizedtesting.SeedInfo.seed([624CA185227AE717]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)




Build Log:
[...truncated 10880 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/init-core-data-001
   [junit4]   2> 1883666 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[624CA185227AE717]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 1883666 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[624CA185227AE717]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 1883671 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1883672 INFO  (Thread-53926) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1883672 INFO  (Thread-53926) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1883773 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.ZkTestServer start zk server on port:39004
   [junit4]   2> 1883773 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1883775 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1883778 INFO  (zkCallback-932-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@53223dd5 name:ZooKeeperConnection Watcher:127.0.0.1:39004 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1883778 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1883779 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1883779 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1883782 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x151bcb0a7500000, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 1883791 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1883792 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1883795 INFO  (zkCallback-933-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@11ab8b57 name:ZooKeeperConnection Watcher:127.0.0.1:39004/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1883795 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1883795 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1883795 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1883797 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1883799 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1883800 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1883802 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1883802 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1883805 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1883805 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1883807 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1883807 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1883808 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1883808 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1883810 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1883810 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1883812 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1883812 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1883813 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1883813 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1883815 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1883815 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1883817 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1883817 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1883818 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1883818 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1883820 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1883820 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1884227 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/control-001/cores/collection1
   [junit4]   2> 1884228 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1884248 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1dbe9a31{/,null,AVAILABLE}
   [junit4]   2> 1884250 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.e.j.s.ServerConnector Started ServerConnector@4c7d1638{HTTP/1.1}{127.0.0.1:57256}
   [junit4]   2> 1884251 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.e.j.s.Server Started @1886938ms
   [junit4]   2> 1884251 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/control-001/cores, hostContext=/, hostPort=57256, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/tempDir-001/control/data}
   [junit4]   2> 1884251 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1884251 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/control-001'
   [junit4]   2> 1884251 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1884251 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1884252 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1884256 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1884261 INFO  (zkCallback-934-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7651a346 name:ZooKeeperConnection Watcher:127.0.0.1:39004/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1884261 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1884262 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1884263 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1884263 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/control-001/solr.xml
   [junit4]   2> 1884279 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/control-001/cores
   [junit4]   2> 1884279 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.CoreContainer New CoreContainer 1253870921
   [junit4]   2> 1884279 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/control-001]
   [junit4]   2> 1884279 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/control-001/lib
   [junit4]   2> 1884280 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1884281 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1884282 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1884282 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1884282 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39004/solr
   [junit4]   2> 1884282 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1884282 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1884282 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1884283 INFO  (zkCallback-936-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@22b16274 name:ZooKeeperConnection Watcher:127.0.0.1:39004 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1884284 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1884285 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1884286 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:57256_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1884289 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x151bcb0a7500003, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 1884290 INFO  (zkCallback-937-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@10d6b753 name:ZooKeeperConnection Watcher:127.0.0.1:39004/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1884290 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:57256_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1884291 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:57256_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1884293 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:57256_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1884296 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:57256_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1884302 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:57256_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1884305 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:57256_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1884307 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:57256_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1884308 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:57256_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1884310 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:57256_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1884312 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:57256_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1884328 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:57256_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57256_
   [junit4]   2> 1884329 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:57256_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:57256_
   [junit4]   2> 1884332 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:57256_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1884334 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:57256_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1884335 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:57256_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1884335 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:57256_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95064534058795012-127.0.0.1:57256_-n_0000000000
   [junit4]   2> 1884336 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:57256_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:57256_
   [junit4]   2> 1884336 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:57256_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1884337 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:57256_    ] o.a.s.c.Overseer Overseer (id=95064534058795012-127.0.0.1:57256_-n_0000000000) starting
   [junit4]   2> 1884348 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:57256_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1884360 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:57256_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1884361 INFO  (OverseerCollectionConfigSetProcessor-95064534058795012-127.0.0.1:57256_-n_0000000000) [n:127.0.0.1:57256_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1884365 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:57256_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1884366 INFO  (OverseerStateUpdate-95064534058795012-127.0.0.1:57256_-n_0000000000) [n:127.0.0.1:57256_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1884394 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:57256_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1884394 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:57256_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1884395 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:57256_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/control-001/cores
   [junit4]   2> 1884397 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:57256_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {loadOnStartup=true, transient=false, config=solrconfig.xml, configSetProperties=configsetprops.json, name=collection1, dataDir=data/, shard=, collection=control_collection, schema=schema.xml, coreNodeName=}
   [junit4]   2> 1884397 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:57256_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/control-001/cores/collection1
   [junit4]   2> 1884397 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:57256_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1884398 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:57256_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1884398 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:57256_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1884398 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1884398 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1884402 INFO  (OverseerStateUpdate-95064534058795012-127.0.0.1:57256_-n_0000000000) [n:127.0.0.1:57256_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:57256_",
   [junit4]   2>   "base_url":"http://127.0.0.1:57256",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "core":"collection1"} current state version: 0
   [junit4]   2> 1884416 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1884417 INFO  (OverseerStateUpdate-95064534058795012-127.0.0.1:57256_-n_0000000000) [n:127.0.0.1:57256_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:57256_",
   [junit4]   2>   "base_url":"http://127.0.0.1:57256",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 1884417 INFO  (OverseerStateUpdate-95064534058795012-127.0.0.1:57256_-n_0000000000) [n:127.0.0.1:57256_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1884418 INFO  (OverseerStateUpdate-95064534058795012-127.0.0.1:57256_-n_0000000000) [n:127.0.0.1:57256_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1884420 INFO  (zkCallback-937-thread-2-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1885417 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1885417 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1885420 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1885420 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1885429 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1885429 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/control-001/cores/collection1'
   [junit4]   2> 1885429 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1885429 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1885431 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1885439 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1885462 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1885491 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1885498 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1885937 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1885945 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1885947 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1886005 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1886013 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1886027 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1886028 WARN  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1886029 WARN  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1886029 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1886030 WARN  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1886030 WARN  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1886031 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1886032 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1886032 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1886032 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/control-001/cores/collection1/data/]
   [junit4]   2> 1886032 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@684e89b2
   [junit4]   2> 1886033 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/control-001/cores/collection1/data
   [junit4]   2> 1886033 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1886033 WARN  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1886034 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/control-001/cores/collection1/data/index
   [junit4]   2> 1886034 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=42, maxMergeAtOnceExplicit=39, maxMergedSegmentMB=37.947265625, floorSegmentMB=1.98828125, forceMergeDeletesPctAllowed=4.5759800137177, segmentsPerTier=18.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.30878684524473454
   [junit4]   2> 1886035 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@7e77d7f4 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6e031509),segFN=segments_1,generation=1}
   [junit4]   2> 1886035 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1886035 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/control-001/cores/collection1/data/
   [junit4]   2> 1886035 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1886038 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1886039 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1886039 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1886039 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1886039 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1886039 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1886039 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1886040 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1886040 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1886040 WARN  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 1886043 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1886045 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1886049 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1886051 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1886055 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/system,/admin/logging,/admin/properties,/admin/plugins,standard,/replication,/admin/luke,/admin/segments,/admin/mbeans,/get,/dump,/config,/update/json,/admin/ping,/update/json/docs,/update/csv,/update,/admin/file,/admin/threads,/schema
   [junit4]   2> 1886056 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1886057 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1886057 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1886058 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1886058 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1886059 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1238564430, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 1886059 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@7e77d7f4 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6e031509),segFN=segments_1,generation=1}
   [junit4]   2> 1886059 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1886059 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@745af3b5[collection1] main
   [junit4]   2> 1886059 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1886061 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1886061 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1886061 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1886071 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1886071 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1886072 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1886072 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1886072 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1886072 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1886073 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1886073 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1886073 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1886073 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1521032547457302528
   [junit4]   2> 1886074 INFO  (searcherExecutor-3534-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:57256_ c:control_collection x:collection1) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@745af3b5[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1886075 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1521032547457302528
   [junit4]   2> 1886075 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1886076 INFO  (coreLoadExecutor-3533-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1886077 INFO  (coreZkRegister-3527-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:57256_ c:control_collection x:collection1) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:57256 collection:control_collection shard:shard1
   [junit4]   2> 1886077 INFO  (coreZkRegister-3527-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:57256_ c:control_collection x:collection1) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1886081 INFO  (coreZkRegister-3527-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:57256_ c:control_collection x:collection1) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95064534058795012-core_node1-n_0000000000
   [junit4]   2> 1886081 INFO  (coreZkRegister-3527-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:57256_ c:control_collection x:collection1) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=180000
   [junit4]   2> 1886083 INFO  (coreZkRegister-3527-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:57256_ c:control_collection x:collection1) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1886083 INFO  (coreZkRegister-3527-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:57256_ c:control_collection x:collection1) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1886083 INFO  (coreZkRegister-3527-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:57256_ c:control_collection x:collection1) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:57256/collection1/
   [junit4]   2> 1886083 INFO  (coreZkRegister-3527-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:57256_ c:control_collection x:collection1) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1886083 INFO  (coreZkRegister-3527-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:57256_ c:control_collection x:collection1) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:57256/collection1/ has no replicas
   [junit4]   2> 1886084 INFO  (OverseerStateUpdate-95064534058795012-127.0.0.1:57256_-n_0000000000) [n:127.0.0.1:57256_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1886085 INFO  (coreZkRegister-3527-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:57256_ c:control_collection x:collection1) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1886085 INFO  (zkCallback-937-thread-2-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1886120 INFO  (coreZkRegister-3527-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:57256_ c:control_collection x:collection1) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1886123 INFO  (coreZkRegister-3527-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:57256_ c:control_collection x:collection1) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:57256/collection1/ shard1
   [junit4]   2> 1886130 INFO  (OverseerStateUpdate-95064534058795012-127.0.0.1:57256_-n_0000000000) [n:127.0.0.1:57256_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:57256",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 1886247 INFO  (zkCallback-937-thread-2-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1886284 INFO  (coreZkRegister-3527-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:57256_ c:control_collection x:collection1) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:57256/collection1/ and leader is http://127.0.0.1:57256/collection1/
   [junit4]   2> 1886284 INFO  (coreZkRegister-3527-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:57256_ c:control_collection x:collection1) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57256
   [junit4]   2> 1886284 INFO  (coreZkRegister-3527-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:57256_ c:control_collection x:collection1) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1886284 INFO  (coreZkRegister-3527-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:57256_ c:control_collection x:collection1) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1886284 INFO  (coreZkRegister-3527-thread-1-processing-r:core_node1 s:shard1 n:127.0.0.1:57256_ c:control_collection x:collection1) [n:127.0.0.1:57256_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1886286 INFO  (OverseerStateUpdate-95064534058795012-127.0.0.1:57256_-n_0000000000) [n:127.0.0.1:57256_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "node_name":"127.0.0.1:57256_",
   [junit4]   2>   "base_url":"http://127.0.0.1:57256",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core":"collection1"} current state version: 3
   [junit4]   2> 1886287 INFO  (OverseerStateUpdate-95064534058795012-127.0.0.1:57256_-n_0000000000) [n:127.0.0.1:57256_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "node_name":"127.0.0.1:57256_",
   [junit4]   2>   "base_url":"http://127.0.0.1:57256",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 1886390 INFO  (zkCallback-937-thread-2-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1886400 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1886401 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1886401 INFO  (zkCallback-939-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5f04cfaa name:ZooKeeperConnection Watcher:127.0.0.1:39004/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1886402 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1886402 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1886402 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1886404 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1886404 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 1886404 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1886406 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1886438 INFO  (zkCallback-940-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7f811746 name:ZooKeeperConnection Watcher:127.0.0.1:39004/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1886439 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1886439 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1886441 INFO  (OverseerStateUpdate-95064534058795012-127.0.0.1:57256_-n_0000000000) [n:127.0.0.1:57256_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 4
   [junit4]   2> 1886441 INFO  (OverseerStateUpdate-95064534058795012-127.0.0.1:57256_-n_0000000000) [n:127.0.0.1:57256_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1886543 INFO  (OverseerStateUpdate-95064534058795012-127.0.0.1:57256_-n_0000000000) [n:127.0.0.1:57256_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 1886880 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/shard-1-001/cores/collection1
   [junit4]   2> 1886881 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/shard-1-001
   [junit4]   2> 1886881 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1886886 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@64ae1e45{/,null,AVAILABLE}
   [junit4]   2> 1886886 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.e.j.s.ServerConnector Started ServerConnector@634577a6{HTTP/1.1}{127.0.0.1:60188}
   [junit4]   2> 1886886 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.e.j.s.Server Started @1889574ms
   [junit4]   2> 1886887 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=60188, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/shard-1-001/cores, hostContext=/, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/tempDir-001/jetty1, solrconfig=solrconfig.xml}
   [junit4]   2> 1886887 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1886887 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/shard-1-001'
   [junit4]   2> 1886887 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1886887 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1886887 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1886893 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1886894 INFO  (zkCallback-941-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5660c652 name:ZooKeeperConnection Watcher:127.0.0.1:39004/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1886894 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1886894 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1886895 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1886895 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/shard-1-001/solr.xml
   [junit4]   2> 1886921 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/shard-1-001/cores
   [junit4]   2> 1886921 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.CoreContainer New CoreContainer 157467320
   [junit4]   2> 1886921 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/shard-1-001]
   [junit4]   2> 1886921 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/shard-1-001/lib
   [junit4]   2> 1886921 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1886923 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1886923 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1886923 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1886923 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39004/solr
   [junit4]   2> 1886923 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1886923 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1886924 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1886925 INFO  (zkCallback-943-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6c8a472d name:ZooKeeperConnection Watcher:127.0.0.1:39004 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1886929 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1886930 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1886935 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:60188_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1886936 INFO  (zkCallback-944-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5bb2f613 name:ZooKeeperConnection Watcher:127.0.0.1:39004/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1886936 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:60188_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1886942 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:60188_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1887965 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:60188_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60188_
   [junit4]   2> 1887966 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:60188_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:60188_
   [junit4]   2> 1887967 INFO  (zkCallback-944-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1887967 INFO  (zkCallback-939-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1887967 INFO  (zkCallback-937-thread-1-processing-n:127.0.0.1:57256_) [n:127.0.0.1:57256_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1887971 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:60188_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1887972 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:60188_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95064534058795017-127.0.0.1:60188_-n_0000000001
   [junit4]   2> 1887973 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:60188_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/95064534058795012-127.0.0.1:57256_-n_0000000000 to know if I could be the leader
   [junit4]   2> 1887986 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:60188_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1887986 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:60188_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1887987 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:60188_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/shard-1-001/cores
   [junit4]   2> 1887988 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:60188_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, collection=collection1, shard=, configSetProperties=configsetprops.json, schema=schema.xml, coreNodeName=, transient=false, config=solrconfig.xml, loadOnStartup=true, dataDir=data/}
   [junit4]   2> 1887988 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:60188_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/shard-1-001/cores/collection1
   [junit4]   2> 1887988 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:60188_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1887990 INFO  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1887990 INFO  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1887991 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:60188_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1887991 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[624CA185227AE717]) [n:127.0.0.1:60188_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1887992 INFO  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 1887992 INFO  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 1887992 INFO  (OverseerStateUpdate-95064534058795012-127.0.0.1:57256_-n_0000000000) [n:127.0.0.1:57256_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:60188_",
   [junit4]   2>   "base_url":"http://127.0.0.1:60188",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "core":"collection1"} current state version: 4
   [junit4]   2> 1887992 INFO  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for collection1 ver 0 
   [junit4]   2> 1887992 INFO  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1887992 INFO  (OverseerStateUpdate-95064534058795012-127.0.0.1:57256_-n_0000000000) [n:127.0.0.1:57256_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:60188_",
   [junit4]   2>   "base_url":"http://127.0.0.1:60188",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 1887992 INFO  (OverseerStateUpdate-95064534058795012-127.0.0.1:57256_-n_0000000000) [n:127.0.0.1:57256_    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=2
   [junit4]   2> 1887992 INFO  (OverseerStateUpdate-95064534058795012-127.0.0.1:57256_-n_0000000000) [n:127.0.0.1:57256_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 1888121 INFO  (OverseerStateUpdate-95064534058795012-127.0.0.1:57256_-n_0000000000) [n:127.0.0.1:57256_    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 1888122 INFO  (zkCallback-944-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1888123 INFO  (zkCallback-944-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_    ] o.a.s.c.c.ZkStateReader Updating data for collection1 from 0 to 1 
   [junit4]   2> 1888992 INFO  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1888992 INFO  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1888993 INFO  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1888993 INFO  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 1888994 INFO  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1888994 INFO  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/shard-1-001/cores/collection1'
   [junit4]   2> 1888994 INFO  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1888994 INFO  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1888996 INFO  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1889013 INFO  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1889036 INFO  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1889083 INFO  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1889101 INFO  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1889476 INFO  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1889520 INFO  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1889522 INFO  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1889530 INFO  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1889534 INFO  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1889536 INFO  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1889537 WARN  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1889537 WARN  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1889537 INFO  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1889538 WARN  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1889538 WARN  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1889539 INFO  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1889539 INFO  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 1889539 INFO  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1889539 INFO  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/shard-1-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_624CA185227AE717-001/shard-1-001/cores/collection1/data/]
   [junit4]   2> 1889539 INFO  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@684e89b2
   [junit4]   2> 1889540 INFO  (coreLoadExecutor-3544-thread-1-processing-n:127.0.0.1:60188_) [n:127.0.0.1:60188_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Nig

[...truncated too long message...]

tions/awholynewstresscollection_collection1_2/leader_elect/shard3/election/95064534058795020-core_node19-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection5_12/leader_elect/shard1/election/95064534058795026-core_node9-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection3_3/leader_elect/shard4/election/95064534058795026-core_node15-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewcollection_0/leader_elect/shard2/election/95064534058795026-core_node15-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection2_3/leader_elect/shard3/election/95064534058795023-core_node16-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection1_4/leader_elect/shard9/election/95064534058795020-core_node2-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection2_4/leader_elect/shard2/election/95064534058795012-core_node9-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection2_5/leader_elect/shard6/election/95064534058795023-core_node16-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection1_5/leader_elect/shard4/election/95064534058795012-core_node16-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection5_5/leader_elect/shard1/election/95064534058795023-core_node23-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection2_2/leader_elect/shard5/election/95064534058795012-core_node28-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewcollection_0/leader_elect/shard4/election/95064534058795026-core_node10-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection0_3/leader_elect/shard2/election/95064534058795017-core_node6-n_0000000001
   [junit4]   2> 	2	/solr/collections/halfdeletedcollection2/leader_elect/shard3/election/95064534058795026-core_node5-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection1_4/leader_elect/shard6/election/95064534058795023-core_node5-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection0_5/leader_elect/shard2/election/95064534058795020-core_node27-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection5_2/leader_elect/shard5/election/95064534058795020-core_node12-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection3_3/leader_elect/shard3/election/95064534058795020-core_node4-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection2_4/leader_elect/shard7/election/95064534058795017-core_node10-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection2_4/leader_elect/shard5/election/95064534058795026-core_node11-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection5_5/leader_elect/shard2/election/95064534058795017-core_node16-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection3_2/leader_elect/shard8/election/95064534058795012-core_node4-n_0000000000
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	88	/solr/collections
   [junit4]   2> 	87	/solr/live_nodes
   [junit4]   2> 	3	/solr/overseer/queue
   [junit4]   2> 	3	/solr/overseer/collection-queue-work
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=CollectionsAPIDistributedZkTest -Dtests.method=test -Dtests.seed=624CA185227AE717 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=ar_SA -Dtests.timezone=Africa/Bissau -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR    526s J0 | CollectionsAPIDistributedZkTest.test <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=59568, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([624CA185227AE717:EA189E5F8C868AEF]:0)
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:60188: Could not find collection : awholynewstresscollection_collection4_2
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([624CA185227AE717]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)
   [junit4]   2> 2409493 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[624CA185227AE717]-worker) [n:127.0.0.1:55878_ c:addReplicaColl s:shard2 r:core_node6 x:addReplicaColl_shard2_replica3] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=DefaultSimilarity, locale=ar_SA, timezone=Africa/Bissau
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=312863408,total=503840768
   [junit4]   2> NOTE: All tests run in this JVM: [TestStressLucene, ReplicationFactorTest, JsonLoaderTest, SignatureUpdateProcessorFactoryTest, TestLuceneMatchVersion, SystemInfoHandlerTest, HdfsNNFailoverTest, TestHighlightDedupGrouping, TestLazyCores, PathHierarchyTokenizerFactoryTest, IndexSchemaTest, SSLMigrationTest, AlternateDirectoryTest, TestValueSourceCache, SOLR749Test, TestRemoteStreaming, FileBasedSpellCheckerTest, TestHighFrequencyDictionaryFactory, CoreAdminCreateDiscoverTest, AsyncMigrateRouteKeyTest, BufferStoreTest, TestConfigSetsAPI, TestMiniSolrCloudClusterSSL, TestInitQParser, ZkNodePropsTest, PreAnalyzedFieldTest, TestSchemaVersionResource, TestClassicSimilarityFactory, DateFieldTest, TestPhraseSuggestions, TestCSVLoader, UUIDUpdateProcessorFallbackTest, DeleteInactiveReplicaTest, TestCustomSort, TestPKIAuthenticationPlugin, RequiredFieldsTest, ExternalFileFieldSortTest, OverseerTaskQueueTest, TestOmitPositions, RulesTest, JavabinLoaderTest, TestMacros, EchoParamsTest, TestRestoreCore, SolrCoreTest, XsltUpdateRequestHandlerTest, DistributedFacetPivotLongTailTest, TestReloadDeadlock, TestDistributedSearch, TestSolrXml, BasicZkTest, TestArbitraryIndexDir, HLLSerializationTest, TestCopyFieldCollectionResource, TestRestManager, TestExactSharedStatsCache, ResponseLogComponentTest, HdfsWriteToMultipleCollectionsTest, TestConfigReload, AnalysisAfterCoreReloadTest, UUIDFieldTest, TestShardHandlerFactory, OutputWriterTest, TestShortCircuitedRequests, SyncSliceTest, HdfsSyncSliceTest, JSONWriterTest, TestExclusionRuleCollectionAccess, CloneFieldUpdateProcessorFactoryTest, DisMaxRequestHandlerTest, OverseerCollectionConfigSetProcessorTest, DistributedSpellCheckComponentTest, ParsingFieldUpdateProcessorsTest, DocumentAnalysisRequestHandlerTest, CloudExitableDirectoryReaderTest, LeaderInitiatedRecoveryOnCommitTest, ZkStateReaderTest, PeerSyncTest, HdfsChaosMonkeyNothingIsSafeTest, QueryElevationComponentTest, TestNoOpRegenerator, DistributedDebugComponentTest, SuggestComponentContextFilterQueryTest, TestRawResponseWriter, TestStressUserVersions, TestPseudoReturnFields, TestFunctionQuery, TestDocBasedVersionConstraints, VersionInfoTest, TestLMDirichletSimilarityFactory, BasicDistributedZkTest, BasicDistributedZk2Test, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest]
   [junit4] Completed [283/555 (1!)] on J0 in 526.70s, 1 test, 1 error <<< FAILURES!

[...truncated 864 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1456: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1010: There were test failures: 555 suites (7 ignored), 2206 tests, 1 error, 33 ignored (18 assumptions) [seed: 624CA185227AE717]

Total time: 190 minutes 21 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1077 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1077/

5 tests failed.
FAILED:  org.apache.solr.hadoop.MorphlineBasicMiniMRTest.mrRun

Error Message:
Failed on local exception: java.io.IOException: Connection reset by peer; Host Details : local host is: "lucene1-us-west/10.41.0.5"; destination host is: "lucene1-us-west.apache.org":42405; 

Stack Trace:
java.io.IOException: Failed on local exception: java.io.IOException: Connection reset by peer; Host Details : local host is: "lucene1-us-west/10.41.0.5"; destination host is: "lucene1-us-west.apache.org":42405; 
	at __randomizedtesting.SeedInfo.seed([6AA3CDACAB3E11E8:64F179A2AAA823E7]:0)
	at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
	at org.apache.hadoop.ipc.Client.call(Client.java:1472)
	at org.apache.hadoop.ipc.Client.call(Client.java:1399)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
	at com.sun.proxy.$Proxy47.getClusterMetrics(Unknown Source)
	at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterMetrics(ApplicationClientProtocolPBClientImpl.java:202)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
	at com.sun.proxy.$Proxy48.getClusterMetrics(Unknown Source)
	at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getYarnClusterMetrics(YarnClientImpl.java:461)
	at org.apache.hadoop.mapred.ResourceMgrDelegate.getClusterMetrics(ResourceMgrDelegate.java:151)
	at org.apache.hadoop.mapred.YARNRunner.getClusterMetrics(YARNRunner.java:179)
	at org.apache.hadoop.mapreduce.Cluster.getClusterStatus(Cluster.java:246)
	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:719)
	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:717)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
	at org.apache.hadoop.mapred.JobClient.getClusterStatus(JobClient.java:717)
	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:645)
	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:608)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.solr.hadoop.MorphlineBasicMiniMRTest.mrRun(MorphlineBasicMiniMRTest.java:363)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Connection reset by peer
	at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
	at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
	at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
	at sun.nio.ch.IOUtil.read(IOUtil.java:197)
	at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:384)
	at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:57)
	at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
	at java.io.FilterInputStream.read(FilterInputStream.java:133)
	at java.io.FilterInputStream.read(FilterInputStream.java:133)
	at org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:513)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:1071)
	at org.apache.hadoop.ipc.Client$Connection.run(Client.java:966)


FAILED:  org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest.test

Error Message:
Failed on local exception: java.io.IOException: Connection reset by peer; Host Details : local host is: "lucene1-us-west/10.41.0.5"; destination host is: "lucene1-us-west.apache.org":42345; 

Stack Trace:
java.io.IOException: Failed on local exception: java.io.IOException: Connection reset by peer; Host Details : local host is: "lucene1-us-west/10.41.0.5"; destination host is: "lucene1-us-west.apache.org":42345; 
	at __randomizedtesting.SeedInfo.seed([6AA3CDACAB3E11E8:E2F7F27605C27C10]:0)
	at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
	at org.apache.hadoop.ipc.Client.call(Client.java:1472)
	at org.apache.hadoop.ipc.Client.call(Client.java:1399)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
	at com.sun.proxy.$Proxy47.getClusterMetrics(Unknown Source)
	at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterMetrics(ApplicationClientProtocolPBClientImpl.java:202)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
	at com.sun.proxy.$Proxy48.getClusterMetrics(Unknown Source)
	at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getYarnClusterMetrics(YarnClientImpl.java:461)
	at org.apache.hadoop.mapred.ResourceMgrDelegate.getClusterMetrics(ResourceMgrDelegate.java:151)
	at org.apache.hadoop.mapred.YARNRunner.getClusterMetrics(YARNRunner.java:179)
	at org.apache.hadoop.mapreduce.Cluster.getClusterStatus(Cluster.java:246)
	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:719)
	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:717)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
	at org.apache.hadoop.mapred.JobClient.getClusterStatus(JobClient.java:717)
	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:645)
	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:608)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest.test(MorphlineGoLiveMiniMRTest.java:402)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:965)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:940)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Connection reset by peer
	at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
	at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
	at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
	at sun.nio.ch.IOUtil.read(IOUtil.java:197)
	at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:384)
	at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:57)
	at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
	at java.io.FilterInputStream.read(FilterInputStream.java:133)
	at java.io.FilterInputStream.read(FilterInputStream.java:133)
	at org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:513)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:1071)
	at org.apache.hadoop.ipc.Client$Connection.run(Client.java:966)


FAILED:  org.apache.solr.hadoop.MorphlineMapperTest.testMapper

Error Message:
Malformed / non-existent locale:  near: {     # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 198     "fmap" : {         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 198         "content-type" : "content_type",         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 198         "content" : "text"     },     # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 160     #  captureAttr : true # default is false     "capture" : [         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 163         #  twitter feed schema         "user_friends_count",         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 164         "user_location",         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 165         "user_description",         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 166         "user_statuses_count",         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 167         "user_followers_count",         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 168         "user_name",         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 169         "user_screen_name",         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 170         "created_at",         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 171         "text",         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 172         "retweet_count",         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 173         "retweeted",         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 174         "in_reply_to_user_id",         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 175         "source",         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 176         "in_reply_to_status_id",         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 177         "media_url_https",         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 178         "expanded_url",         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 181         #  file metadata            "file_download_url",         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 182         "file_upload_url",         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 183         "file_scheme",         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 184         "file_host",         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 185         "file_port",         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 186         "file_path",         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 187         "file_name",         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 188         "file_length",         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 189         "file_last_modified",         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 190         "file_owner",         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 191         "file_group",         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 192         "file_permissions_user",         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 193         "file_permissions_group",         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 194         "file_permissions_other",         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 195         "file_permissions_stickybit"     ],     # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 199     #  rename "content" field to "text" fields     "dateFormats" : [         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 199         "yyyy-MM-dd'T'HH:mm:ss",         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 199         "yyyy-MM-dd"     ],     # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 158     "solrLocator" : {         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 158         "collection" : "collection1"     },     # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 202     "lowernames" : true,     # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 207     #  Tika parsers to be registered. If multiple parsers support the same MIME type,      #  the parser is chosen that is closest to the bottom in this list:     "parsers" : [         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 208         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 208             "parser" : "org.apache.tika.parser.asm.ClassParser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 211         #  { parser : org.apache.tika.parser.AutoDetectParser }                               #  { parser : org.gagravarr.tika.OggParser, additionalSupportedMimeTypes : [audio/ogg] }         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 211             "parser" : "org.gagravarr.tika.FlacParser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 212         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 212             "parser" : "org.apache.tika.parser.audio.AudioParser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 213         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 213             "parser" : "org.apache.tika.parser.audio.MidiParser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 214         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 214             "parser" : "org.apache.tika.parser.crypto.Pkcs7Parser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 215         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 215             "parser" : "org.apache.tika.parser.dwg.DWGParser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 216         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 216             "parser" : "org.apache.tika.parser.epub.EpubParser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 217         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 217             "parser" : "org.apache.tika.parser.executable.ExecutableParser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 218         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 218             "parser" : "org.apache.tika.parser.feed.FeedParser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 219         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 219             "parser" : "org.apache.tika.parser.font.AdobeFontMetricParser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 220         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 220             "parser" : "org.apache.tika.parser.font.TrueTypeParser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 221         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 221             "parser" : "org.apache.tika.parser.xml.XMLParser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 222         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 222             "parser" : "org.apache.tika.parser.html.HtmlParser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 223         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 223             "parser" : "org.apache.tika.parser.image.ImageParser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 224         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 224             "parser" : "org.apache.tika.parser.image.PSDParser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 225         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 225             "parser" : "org.apache.tika.parser.image.TiffParser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 226         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 226             "parser" : "org.apache.tika.parser.iptc.IptcAnpaParser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 227         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 227             "parser" : "org.apache.tika.parser.iwork.IWorkPackageParser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 228         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 228             "parser" : "org.apache.tika.parser.jpeg.JpegParser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 229         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 229             "parser" : "org.apache.tika.parser.mail.RFC822Parser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 230         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 230             "additionalSupportedMimeTypes" : [                 # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 230                 "message/x-emlx"             ],             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 230             "parser" : "org.apache.tika.parser.mbox.MboxParser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 231         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 231             "parser" : "org.apache.tika.parser.microsoft.OfficeParser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 232         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 232             "parser" : "org.apache.tika.parser.microsoft.TNEFParser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 233         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 233             "parser" : "org.apache.tika.parser.microsoft.ooxml.OOXMLParser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 234         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 234             "parser" : "org.apache.tika.parser.mp3.Mp3Parser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 235         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 235             "parser" : "org.apache.tika.parser.mp4.MP4Parser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 236         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 236             "parser" : "org.apache.tika.parser.hdf.HDFParser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 237         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 237             "parser" : "org.apache.tika.parser.netcdf.NetCDFParser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 238         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 238             "parser" : "org.apache.tika.parser.odf.OpenDocumentParser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 239         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 239             "parser" : "org.apache.tika.parser.pdf.PDFParser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 240         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 240             "parser" : "org.apache.tika.parser.pkg.CompressorParser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 241         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 241             "parser" : "org.apache.tika.parser.pkg.PackageParser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 242         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 242             "parser" : "org.apache.tika.parser.rtf.RTFParser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 243         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 243             "parser" : "org.apache.tika.parser.txt.TXTParser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 244         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 244             "parser" : "org.apache.tika.parser.video.FLVParser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 245         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 245             "parser" : "org.apache.tika.parser.xml.DcXMLParser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 246         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 246             "parser" : "org.apache.tika.parser.xml.FictionBookParser"         },         # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 247         {             # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 247             "parser" : "org.apache.tika.parser.chm.ChmParser"         }     ],     # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 201     #  various java.text.SimpleDateFormat     #  xpath : "/xhtml:html/xhtml:body/xhtml:div/descendant:node()"     "uprefix" : "ignored_" }

Stack Trace:
org.kitesdk.morphline.api.MorphlineCompilationException: Malformed / non-existent locale:  near: {
    # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 198
    "fmap" : {
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 198
        "content-type" : "content_type",
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 198
        "content" : "text"
    },
    # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 160
    #  captureAttr : true # default is false
    "capture" : [
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 163
        #  twitter feed schema
        "user_friends_count",
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 164
        "user_location",
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 165
        "user_description",
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 166
        "user_statuses_count",
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 167
        "user_followers_count",
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 168
        "user_name",
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 169
        "user_screen_name",
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 170
        "created_at",
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 171
        "text",
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 172
        "retweet_count",
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 173
        "retweeted",
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 174
        "in_reply_to_user_id",
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 175
        "source",
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 176
        "in_reply_to_status_id",
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 177
        "media_url_https",
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 178
        "expanded_url",
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 181
        #  file metadata   
        "file_download_url",
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 182
        "file_upload_url",
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 183
        "file_scheme",
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 184
        "file_host",
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 185
        "file_port",
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 186
        "file_path",
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 187
        "file_name",
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 188
        "file_length",
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 189
        "file_last_modified",
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 190
        "file_owner",
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 191
        "file_group",
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 192
        "file_permissions_user",
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 193
        "file_permissions_group",
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 194
        "file_permissions_other",
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 195
        "file_permissions_stickybit"
    ],
    # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 199
    #  rename "content" field to "text" fields
    "dateFormats" : [
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 199
        "yyyy-MM-dd'T'HH:mm:ss",
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 199
        "yyyy-MM-dd"
    ],
    # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 158
    "solrLocator" : {
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 158
        "collection" : "collection1"
    },
    # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 202
    "lowernames" : true,
    # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 207
    #  Tika parsers to be registered. If multiple parsers support the same MIME type, 
    #  the parser is chosen that is closest to the bottom in this list:
    "parsers" : [
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 208
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 208
            "parser" : "org.apache.tika.parser.asm.ClassParser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 211
        #  { parser : org.apache.tika.parser.AutoDetectParser }                      
        #  { parser : org.gagravarr.tika.OggParser, additionalSupportedMimeTypes : [audio/ogg] }
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 211
            "parser" : "org.gagravarr.tika.FlacParser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 212
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 212
            "parser" : "org.apache.tika.parser.audio.AudioParser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 213
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 213
            "parser" : "org.apache.tika.parser.audio.MidiParser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 214
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 214
            "parser" : "org.apache.tika.parser.crypto.Pkcs7Parser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 215
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 215
            "parser" : "org.apache.tika.parser.dwg.DWGParser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 216
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 216
            "parser" : "org.apache.tika.parser.epub.EpubParser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 217
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 217
            "parser" : "org.apache.tika.parser.executable.ExecutableParser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 218
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 218
            "parser" : "org.apache.tika.parser.feed.FeedParser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 219
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 219
            "parser" : "org.apache.tika.parser.font.AdobeFontMetricParser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 220
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 220
            "parser" : "org.apache.tika.parser.font.TrueTypeParser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 221
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 221
            "parser" : "org.apache.tika.parser.xml.XMLParser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 222
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 222
            "parser" : "org.apache.tika.parser.html.HtmlParser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 223
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 223
            "parser" : "org.apache.tika.parser.image.ImageParser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 224
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 224
            "parser" : "org.apache.tika.parser.image.PSDParser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 225
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 225
            "parser" : "org.apache.tika.parser.image.TiffParser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 226
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 226
            "parser" : "org.apache.tika.parser.iptc.IptcAnpaParser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 227
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 227
            "parser" : "org.apache.tika.parser.iwork.IWorkPackageParser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 228
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 228
            "parser" : "org.apache.tika.parser.jpeg.JpegParser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 229
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 229
            "parser" : "org.apache.tika.parser.mail.RFC822Parser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 230
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 230
            "additionalSupportedMimeTypes" : [
                # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 230
                "message/x-emlx"
            ],
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 230
            "parser" : "org.apache.tika.parser.mbox.MboxParser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 231
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 231
            "parser" : "org.apache.tika.parser.microsoft.OfficeParser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 232
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 232
            "parser" : "org.apache.tika.parser.microsoft.TNEFParser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 233
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 233
            "parser" : "org.apache.tika.parser.microsoft.ooxml.OOXMLParser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 234
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 234
            "parser" : "org.apache.tika.parser.mp3.Mp3Parser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 235
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 235
            "parser" : "org.apache.tika.parser.mp4.MP4Parser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 236
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 236
            "parser" : "org.apache.tika.parser.hdf.HDFParser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 237
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 237
            "parser" : "org.apache.tika.parser.netcdf.NetCDFParser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 238
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 238
            "parser" : "org.apache.tika.parser.odf.OpenDocumentParser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 239
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 239
            "parser" : "org.apache.tika.parser.pdf.PDFParser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 240
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 240
            "parser" : "org.apache.tika.parser.pkg.CompressorParser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 241
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 241
            "parser" : "org.apache.tika.parser.pkg.PackageParser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 242
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 242
            "parser" : "org.apache.tika.parser.rtf.RTFParser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 243
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 243
            "parser" : "org.apache.tika.parser.txt.TXTParser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 244
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 244
            "parser" : "org.apache.tika.parser.video.FLVParser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 245
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 245
            "parser" : "org.apache.tika.parser.xml.DcXMLParser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 246
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 246
            "parser" : "org.apache.tika.parser.xml.FictionBookParser"
        },
        # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 247
        {
            # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 247
            "parser" : "org.apache.tika.parser.chm.ChmParser"
        }
    ],
    # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J2/temp/solr.hadoop.MorphlineMapperTest_6AA3CDACAB3E11E8-001/tempDir-001/test-morphlines/solrCellDocumentTypes.conf: 201
    #  various java.text.SimpleDateFormat
    #  xpath : "/xhtml:html/xhtml:body/xhtml:div/descendant:node()"
    "uprefix" : "ignored_"
}
	at __randomizedtesting.SeedInfo.seed([6AA3CDACAB3E11E8:A0D7BDA0C0015899]:0)
	at org.apache.solr.morphlines.cell.SolrCellBuilder$SolrCell.getLocale(SolrCellBuilder.java:340)
	at org.apache.solr.morphlines.cell.SolrCellBuilder$SolrCell.<init>(SolrCellBuilder.java:166)
	at org.apache.solr.morphlines.cell.SolrCellBuilder.build(SolrCellBuilder.java:90)
	at org.kitesdk.morphline.base.AbstractCommand.buildCommand(AbstractCommand.java:302)
	at org.kitesdk.morphline.base.AbstractCommand.buildCommandChain(AbstractCommand.java:249)
	at org.kitesdk.morphline.stdlib.TryRulesBuilder$TryRules.<init>(TryRulesBuilder.java:83)
	at org.kitesdk.morphline.stdlib.TryRulesBuilder.build(TryRulesBuilder.java:59)
	at org.kitesdk.morphline.base.AbstractCommand.buildCommand(AbstractCommand.java:302)
	at org.kitesdk.morphline.base.AbstractCommand.buildCommandChain(AbstractCommand.java:249)
	at org.kitesdk.morphline.stdlib.Pipe.<init>(Pipe.java:46)
	at org.kitesdk.morphline.stdlib.PipeBuilder.build(PipeBuilder.java:40)
	at org.kitesdk.morphline.base.Compiler.compile(Compiler.java:126)
	at org.kitesdk.morphline.base.Compiler.compile(Compiler.java:55)
	at org.apache.solr.hadoop.morphline.MorphlineMapRunner.<init>(MorphlineMapRunner.java:158)
	at org.apache.solr.hadoop.morphline.MorphlineMapper.setup(MorphlineMapper.java:76)
	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142)
	at org.apache.hadoop.mrunit.mapreduce.MapDriver.run(MapDriver.java:137)
	at org.apache.solr.hadoop.MorphlineMapperTest.testMapper(MorphlineMapperTest.java:69)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.IllformedLocaleException: Empty subtag [at index 0]
	at java.util.Locale$Builder.setLanguageTag(Locale.java:2311)
	at org.apache.solr.morphlines.cell.SolrCellBuilder$SolrCell.getLocale(SolrCellBuilder.java:338)
	... 56 more


FAILED:  junit.framework.TestSuite.org.apache.solr.hadoop.MorphlineMapperTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.hadoop.MorphlineMapperTest:     1) Thread[id=14, name=Thread-2, state=TIMED_WAITING, group=TGRP-MorphlineMapperTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1033)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1326)         at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:282)         at org.apache.solr.hadoop.HeartBeater.run(HeartBeater.java:109)

Stack Trace:
com.carrotsearch.ra

[...truncated too long message...]

E11E8]-worker) [n:127.0.0.1:51573_ c:collection1 s:shard2 r:core_node5 x:collection1] o.a.h.h.s.n.FSNamesystem Stopping services started for standby state
   [junit4]   2> 1679428 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[6AA3CDACAB3E11E8]-worker) [n:127.0.0.1:51573_ c:collection1 s:shard2 r:core_node5 x:collection1] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
   [junit4]   2> 1679530 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[6AA3CDACAB3E11E8]-worker) [n:127.0.0.1:51573_ c:collection1 s:shard2 r:core_node5 x:collection1] o.a.h.m.i.MetricsSystemImpl Stopping JobHistoryServer metrics system...
   [junit4]   2> 1679533 ERROR (SUITE-MorphlineGoLiveMiniMRTest-seed#[6AA3CDACAB3E11E8]-worker) [n:127.0.0.1:51573_ c:collection1 s:shard2 r:core_node5 x:collection1] o.a.h.m.l.MethodMetric Error invoking method getBlocksTotal
   [junit4]   2> java.lang.reflect.InvocationTargetException
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MethodMetric$2.snapshot(MethodMetric.java:111)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MethodMetric.snapshot(MethodMetric.java:144)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MetricsRegistry.snapshot(MetricsRegistry.java:387)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1.getMetrics(MetricsSourceBuilder.java:79)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:195)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:172)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:151)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getClassName(DefaultMBeanServerInterceptor.java:1804)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
   [junit4]   2> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
   [junit4]   2> 	at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:227)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:461)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest.teardownClass(MorphlineGoLiveMiniMRTest.java:236)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 1679554 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[6AA3CDACAB3E11E8]-worker) [n:127.0.0.1:51573_ c:collection1 s:shard2 r:core_node5 x:collection1] o.a.h.m.i.MetricsSystemImpl JobHistoryServer metrics system stopped.
   [junit4]   2> 1679555 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[6AA3CDACAB3E11E8]-worker) [n:127.0.0.1:51573_ c:collection1 s:shard2 r:core_node5 x:collection1] o.a.h.m.i.MetricsSystemImpl JobHistoryServer metrics system shutdown complete.
   [junit4]   2> 1679556 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[6AA3CDACAB3E11E8]-worker) [n:127.0.0.1:51573_ c:collection1 s:shard2 r:core_node5 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J1/temp/solr.hadoop.MorphlineGoLiveMiniMRTest_6AA3CDACAB3E11E8-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54), sim=DefaultSimilarity, locale=sr-Latn, timezone=Pacific/Efate
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=6,free=215294376,total=305659904
   [junit4]   2> NOTE: All tests run in this JVM: [MorphlineGoLiveMiniMRTest]
   [junit4] Completed [7/7 (3!)] on J1 in 1681.43s, 2 tests, 1 error <<< FAILURES!

[...truncated 14 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:253: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:469: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:534: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1444: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1000: There were test failures: 7 suites, 35 tests, 2 suite-level errors, 3 errors, 1 ignored [seed: 6AA3CDACAB3E11E8]

Total time: 312 minutes 57 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1076 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1076/

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=14907, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=14907, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:32961: Could not find collection : awholynewstresscollection_collection1_0
	at __randomizedtesting.SeedInfo.seed([FD3FB35516E95D91]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:576)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)


FAILED:  org.apache.solr.handler.TestReplicationHandler.doTestStressReplication

Error Message:
timed out waiting for collection1 startAt time to exceed: Sun Jan 17 04:15:23 GMT+00:00 2016

Stack Trace:
java.lang.AssertionError: timed out waiting for collection1 startAt time to exceed: Sun Jan 17 04:15:23 GMT+00:00 2016
	at __randomizedtesting.SeedInfo.seed([FD3FB35516E95D91:2694B39313C13422]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.handler.TestReplicationHandler.watchCoreStartAt(TestReplicationHandler.java:1419)
	at org.apache.solr.handler.TestReplicationHandler.doTestStressReplication(TestReplicationHandler.java:771)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10867 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/init-core-data-001
   [junit4]   2> 1921020 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[FD3FB35516E95D91]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1921020 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[FD3FB35516E95D91]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 1921023 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1921024 INFO  (Thread-9278) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1921024 INFO  (Thread-9278) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1921124 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.ZkTestServer start zk server on port:33675
   [junit4]   2> 1921124 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1921125 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1921127 INFO  (zkCallback-972-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4068ba04 name:ZooKeeperConnection Watcher:127.0.0.1:33675 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1921128 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1921128 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1921128 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1921131 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1921131 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1921133 INFO  (zkCallback-973-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7f7fb059 name:ZooKeeperConnection Watcher:127.0.0.1:33675/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1921133 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1921133 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1921134 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1921136 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1921137 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1921139 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1921141 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1921141 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1921144 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1921144 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1921147 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1921147 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1921149 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1921149 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1921151 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1921152 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1921154 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1921154 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1921156 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1921156 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1921158 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1921158 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1921160 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1921161 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1921163 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1921163 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1921165 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1921165 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1921492 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/control-001/cores/collection1
   [junit4]   2> 1921493 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1921495 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7b4805c2{/,null,AVAILABLE}
   [junit4]   2> 1921498 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.e.j.s.ServerConnector Started ServerConnector@607eb588{HTTP/1.1}{127.0.0.1:32961}
   [junit4]   2> 1921498 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.e.j.s.Server Started @1924461ms
   [junit4]   2> 1921498 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/control-001/cores, hostPort=32961, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/tempDir-001/control/data}
   [junit4]   2> 1921498 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1921498 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/control-001'
   [junit4]   2> 1921498 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1921499 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1921499 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1921499 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1921504 INFO  (zkCallback-974-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7910ea7c name:ZooKeeperConnection Watcher:127.0.0.1:33675/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1921505 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1921505 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1921506 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1921506 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/control-001/solr.xml
   [junit4]   2> 1921523 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/control-001/cores
   [junit4]   2> 1921523 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.CoreContainer New CoreContainer 1954882951
   [junit4]   2> 1921523 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/control-001]
   [junit4]   2> 1921523 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/control-001/lib
   [junit4]   2> 1921524 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1921525 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1921526 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1921526 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1921526 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33675/solr
   [junit4]   2> 1921527 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1921527 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1921527 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1921529 INFO  (zkCallback-976-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@bd0eaa6 name:ZooKeeperConnection Watcher:127.0.0.1:33675 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1921529 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1921529 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1921531 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:32961_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1921532 INFO  (zkCallback-977-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2adf989a name:ZooKeeperConnection Watcher:127.0.0.1:33675/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1921532 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:32961_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1921533 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:32961_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1921535 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:32961_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1921540 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:32961_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1921542 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:32961_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1921544 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:32961_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1921546 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:32961_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1921548 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:32961_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1921550 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:32961_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1921551 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:32961_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1921553 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:32961_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32961_
   [junit4]   2> 1921553 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:32961_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:32961_
   [junit4]   2> 1921555 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:32961_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1921557 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:32961_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1921558 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:32961_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1921559 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:32961_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95224020988461060-127.0.0.1:32961_-n_0000000000
   [junit4]   2> 1921559 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:32961_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:32961_
   [junit4]   2> 1921559 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:32961_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1921560 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:32961_    ] o.a.s.c.Overseer Overseer (id=95224020988461060-127.0.0.1:32961_-n_0000000000) starting
   [junit4]   2> 1921562 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:32961_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1921570 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:32961_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1921571 INFO  (OverseerCollectionConfigSetProcessor-95224020988461060-127.0.0.1:32961_-n_0000000000) [n:127.0.0.1:32961_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1921571 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:32961_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1921572 INFO  (OverseerStateUpdate-95224020988461060-127.0.0.1:32961_-n_0000000000) [n:127.0.0.1:32961_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1921609 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:32961_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1921609 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:32961_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1921610 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:32961_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/control-001/cores
   [junit4]   2> 1921612 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:32961_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {transient=false, dataDir=data/, configSetProperties=configsetprops.json, shard=, coreNodeName=, schema=schema.xml, loadOnStartup=true, config=solrconfig.xml, name=collection1, collection=control_collection}
   [junit4]   2> 1921612 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:32961_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/control-001/cores/collection1
   [junit4]   2> 1921612 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:32961_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1921613 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:32961_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1921613 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:32961_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1921613 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1921613 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1921615 INFO  (OverseerStateUpdate-95224020988461060-127.0.0.1:32961_-n_0000000000) [n:127.0.0.1:32961_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:32961_",
   [junit4]   2>   "base_url":"http://127.0.0.1:32961"} current state version: 0
   [junit4]   2> 1921616 INFO  (OverseerStateUpdate-95224020988461060-127.0.0.1:32961_-n_0000000000) [n:127.0.0.1:32961_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:32961_",
   [junit4]   2>   "base_url":"http://127.0.0.1:32961"}
   [junit4]   2> 1921616 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1921616 INFO  (OverseerStateUpdate-95224020988461060-127.0.0.1:32961_-n_0000000000) [n:127.0.0.1:32961_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1921617 INFO  (OverseerStateUpdate-95224020988461060-127.0.0.1:32961_-n_0000000000) [n:127.0.0.1:32961_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1921619 INFO  (zkCallback-977-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1922617 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1922617 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1922618 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1922618 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 1922619 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1922619 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/control-001/cores/collection1'
   [junit4]   2> 1922619 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1922619 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1922621 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1922628 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1922642 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1922672 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1922680 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1922947 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1922955 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1922957 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1922966 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1922969 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1922971 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1922972 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1922974 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1922974 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1922974 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1922974 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/control-001/cores/collection1/data/]
   [junit4]   2> 1922974 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@166bcf1a
   [junit4]   2> 1922975 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/control-001/cores/collection1/data
   [junit4]   2> 1922975 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1922975 WARN  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1922975 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/control-001/cores/collection1/data/index
   [junit4]   2> 1922975 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=16, maxMergeAtOnceExplicit=39, maxMergedSegmentMB=52.0048828125, floorSegmentMB=0.50390625, forceMergeDeletesPctAllowed=6.93694183714036, segmentsPerTier=45.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.2604247779116181
   [junit4]   2> 1922976 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@76cbaaaa lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3531bbd4),segFN=segments_1,generation=1}
   [junit4]   2> 1922976 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1922977 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/control-001/cores/collection1/data/
   [junit4]   2> 1922977 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1922980 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1922980 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1922980 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1922980 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1922980 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1922980 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1922981 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1922981 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1922981 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1922981 WARN  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 1922984 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1922985 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1922986 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1922988 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1922991 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/properties,/schema,/admin/segments,/update/json,/config,/admin/threads,/update/json/docs,standard,/admin/luke,/admin/plugins,/update/csv,/update,/replication,/admin/logging,/admin/mbeans,/dump,/get,/admin/file,/admin/system,/admin/ping
   [junit4]   2> 1922992 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1922993 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1922993 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1922994 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1922994 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1922994 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=18, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 1922995 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@76cbaaaa lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3531bbd4),segFN=segments_1,generation=1}
   [junit4]   2> 1922995 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1922995 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@760752f2[collection1] main
   [junit4]   2> 1922995 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 1922996 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1922996 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1922996 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1922996 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1922996 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1922997 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1922997 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1922997 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1922997 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1922998 INFO  (searcherExecutor-3359-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@760752f2[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1922998 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1922998 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1922998 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1922998 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1523584337884217344
   [junit4]   2> 1923000 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1523584337884217344
   [junit4]   2> 1923000 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1923001 INFO  (coreLoadExecutor-3358-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1923001 INFO  (coreZkRegister-3352-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:32961 collection:control_collection shard:shard1
   [junit4]   2> 1923002 INFO  (coreZkRegister-3352-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1923006 INFO  (coreZkRegister-3352-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95224020988461060-core_node1-n_0000000000
   [junit4]   2> 1923006 INFO  (coreZkRegister-3352-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=180000
   [junit4]   2> 1923008 INFO  (coreZkRegister-3352-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1923008 INFO  (coreZkRegister-3352-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1923008 INFO  (coreZkRegister-3352-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:32961/collection1/
   [junit4]   2> 1923008 INFO  (OverseerStateUpdate-95224020988461060-127.0.0.1:32961_-n_0000000000) [n:127.0.0.1:32961_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1923008 INFO  (coreZkRegister-3352-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1923008 INFO  (coreZkRegister-3352-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:32961/collection1/ has no replicas
   [junit4]   2> 1923009 INFO  (coreZkRegister-3352-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1923011 INFO  (coreZkRegister-3352-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1923013 INFO  (coreZkRegister-3352-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:32961/collection1/ shard1
   [junit4]   2> 1923014 INFO  (OverseerStateUpdate-95224020988461060-127.0.0.1:32961_-n_0000000000) [n:127.0.0.1:32961_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:32961",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1923115 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1923116 INFO  (zkCallback-977-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1923117 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1923118 INFO  (zkCallback-979-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4af05ce2 name:ZooKeeperConnection Watcher:127.0.0.1:33675/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1923118 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1923119 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1923119 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1923121 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1923121 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 1923121 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1923121 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1923122 INFO  (zkCallback-980-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@70973c50 name:ZooKeeperConnection Watcher:127.0.0.1:33675/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1923122 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1923122 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1923124 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x1524dbe191a0006, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 1923125 INFO  (OverseerStateUpdate-95224020988461060-127.0.0.1:32961_-n_0000000000) [n:127.0.0.1:32961_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 2
   [junit4]   2> 1923125 INFO  (OverseerStateUpdate-95224020988461060-127.0.0.1:32961_-n_0000000000) [n:127.0.0.1:32961_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1923164 INFO  (coreZkRegister-3352-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:32961/collection1/ and leader is http://127.0.0.1:32961/collection1/
   [junit4]   2> 1923164 INFO  (coreZkRegister-3352-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:32961
   [junit4]   2> 1923164 INFO  (coreZkRegister-3352-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1923164 INFO  (coreZkRegister-3352-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1923164 INFO  (coreZkRegister-3352-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:32961_) [n:127.0.0.1:32961_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1923167 INFO  (OverseerStateUpdate-95224020988461060-127.0.0.1:32961_-n_0000000000) [n:127.0.0.1:32961_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:32961_",
   [junit4]   2>   "base_url":"http://127.0.0.1:32961"} current state version: 2
   [junit4]   2> 1923167 INFO  (OverseerStateUpdate-95224020988461060-127.0.0.1:32961_-n_0000000000) [n:127.0.0.1:32961_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:32961_",
   [junit4]   2>   "base_url":"http://127.0.0.1:32961"}
   [junit4]   2> 1923168 INFO  (OverseerStateUpdate-95224020988461060-127.0.0.1:32961_-n_0000000000) [n:127.0.0.1:32961_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 1923271 INFO  (zkCallback-977-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1923272 INFO  (zkCallback-979-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1923431 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/shard-1-001/cores/collection1
   [junit4]   2> 1923432 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/shard-1-001
   [junit4]   2> 1923432 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1923434 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@17b82644{/,null,AVAILABLE}
   [junit4]   2> 1923434 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.e.j.s.ServerConnector Started ServerConnector@4cfe2598{HTTP/1.1}{127.0.0.1:60979}
   [junit4]   2> 1923434 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.e.j.s.Server Started @1926397ms
   [junit4]   2> 1923435 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, hostPort=60979, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/shard-1-001/cores, solrconfig=solrconfig.xml, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/tempDir-001/jetty1}
   [junit4]   2> 1923435 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1923435 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/shard-1-001'
   [junit4]   2> 1923435 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1923435 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1923435 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1923436 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1923437 INFO  (zkCallback-981-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@214fa40f name:ZooKeeperConnection Watcher:127.0.0.1:33675/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1923438 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1923438 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1923439 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1923439 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/shard-1-001/solr.xml
   [junit4]   2> 1923453 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/shard-1-001/cores
   [junit4]   2> 1923454 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.CoreContainer New CoreContainer 544629357
   [junit4]   2> 1923454 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/shard-1-001]
   [junit4]   2> 1923454 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/shard-1-001/lib
   [junit4]   2> 1923454 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1923455 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1923456 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1923456 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1923456 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33675/solr
   [junit4]   2> 1923456 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1923456 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1923457 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1923458 INFO  (zkCallback-983-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5975ba5e name:ZooKeeperConnection Watcher:127.0.0.1:33675 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1923458 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1923458 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1923460 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:60979_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1923461 INFO  (zkCallback-984-thread-1-processing-n:127.0.0.1:60979_) [n:127.0.0.1:60979_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1df9123e name:ZooKeeperConnection Watcher:127.0.0.1:33675/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1923461 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:60979_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1923466 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:60979_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1924472 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:60979_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60979_
   [junit4]   2> 1924472 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:60979_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:60979_
   [junit4]   2> 1924474 INFO  (zkCallback-984-thread-1-processing-n:127.0.0.1:60979_) [n:127.0.0.1:60979_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1924474 INFO  (zkCallback-977-thread-1-processing-n:127.0.0.1:32961_) [n:127.0.0.1:32961_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1924474 INFO  (zkCallback-979-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1924475 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:60979_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1924476 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:60979_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95224020988461065-127.0.0.1:60979_-n_0000000001
   [junit4]   2> 1924477 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:60979_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/95224020988461060-127.0.0.1:32961_-n_0000000000 to know if I could be the leader
   [junit4]   2> 1924489 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:60979_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1924489 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:60979_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1924490 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:60979_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/shard-1-001/cores
   [junit4]   2> 1924491 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:60979_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {dataDir=data/, configSetProperties=configsetprops.json, config=solrconfig.xml, coreNodeName=, collection=collection1, name=collection1, loadOnStartup=true, schema=schema.xml, transient=false, shard=}
   [junit4]   2> 1924491 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:60979_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/shard-1-001/cores/collection1
   [junit4]   2> 1924491 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:60979_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1924493 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:60979_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1924493 INFO  (coreLoadExecutor-3369-thread-1-processing-n:127.0.0.1:60979_) [n:127.0.0.1:60979_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1924493 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[FD3FB35516E95D91]) [n:127.0.0.1:60979_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1924493 INFO  (coreLoadExecutor-3369-thread-1-processing-n:127.0.0.1:60979_) [n:127.0.0.1:60979_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1924494 INFO  (coreLoadExecutor-3369-thread-1-processing-n:127.0.0.1:60979_) [n:127.0.0.1:60979_ c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 1924495 INFO  (coreLoadExecutor-3369-thread-1-processing-n:127.0.0.1:60979_) [n:127.0.0.1:60979_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch [collection1]
   [junit4]   2> 1924495 INFO  (OverseerStateUpdate-95224020988461060-127.0.0.1:32961_-n_0000000000) [n:127.0.0.1:32961_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:60979_",
   [junit4]   2>   "base_url":"http://127.0.0.1:60979"} current state version: 3
   [junit4]   2> 1924495 INFO  (coreLoadExecutor-3369-thread-1-processing-n:127.0.0.1:60979_) [n:127.0.0.1:60979_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for [collection1] ver [0]
   [junit4]   2> 1924495 INFO  (coreLoadExecutor-3369-thread-1-processing-n:127.0.0.1:60979_) [n:127.0.0.1:60979_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1924495 INFO  (OverseerStateUpdate-95224020988461060-127.0.0.1:32961_-n_0000000000) [n:127.0.0.1:32961_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:60979_",
   [junit4]   2>   "base_url":"http://127.0.0.1:60979"}
   [junit4]   2> 1924495 INFO  (OverseerStateUpdate-95224020988461060-127.0.0.1:32961_-n_0000000000) [n:127.0.0.1:32961_    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=2
   [junit4]   2> 1924495 INFO  (OverseerStateUpdate-95224020988461060-127.0.0.1:32961_-n_0000000000) [n:127.0.0.1:32961_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 1924598 INFO  (OverseerStateUpdate-95224020988461060-127.0.0.1:32961_-n_0000000000) [n:127.0.0.1:32961_    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 1924599 INFO  (zkCallback-984-thread-1-processing-n:127.0.0.1:60979_) [n:127.0.0.1:60979_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1924601 INFO  (zkCallback-984-thread-1-processing-n:127.0.0.1:60979_) [n:127.0.0.1:60979_    ] o.a.s.c.c.ZkStateReader Updating data for [collection1] from [0] to [1]
   [junit4]   2> 1925495 INFO  (coreLoadExecutor-3369-thread-1-processing-n:127.0.0.1:60979_) [n:127.0.0.1:60979_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1925495 INFO  (coreLoadExecutor-3369-thread-1-processing-n:127.0.0.1:60979_) [n:127.0.0.1:60979_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1925496 INFO  (coreLoadExecutor-3369-thread-1-processing-n:127.0.0.1:60979_) [n:127.0.0.1:60979_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1925496 INFO  (coreLoadExecutor-3369-thread-1-processing-n:127.0.0.1:60979_) [n:127.0.0.1:60979_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/collection1]
   [junit4]   2> 1925497 INFO  (coreLoadExecutor-3369-thread-1-processing-n:127.0.0.1:60979_) [n:127.0.0.1:60979_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/collection1] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1925497 INFO  (coreLoadExecutor-3369-thread-1-processing-n:127.0.0.1:60979_) [n:127.0.0.1:60979_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_FD3FB35516E95D91-001/shard-1-001/cores/collection1'
   [junit4]   2> 1925497 INFO  (coreLoadExecutor-3369-thread-1-processing-n:127.0.0.1:60979_) [n:127.0.0.1:60979_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1925497 INFO  (coreLoadExecutor-3369-thread-1-processing-n:127.0.0.1:60979_) [n:127.0.0.1:60979_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1925499 INFO  (coreLoadExecutor-3369-thread-1-processing-n:127.0.0.1:60979_) [n:127.0.0.1:60979_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1925506 INFO  (coreLoadExecutor-3369-thread-1-processing-n:127.0.0.1:60979_) [n:127.0.0.1:60979_ c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1925521 INFO  (coreLoadExecutor-3369-thread-1-processing-n:127.0.0.1:60979_) [n:127.0.0.1:60979_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1925550 INFO  (coreLoadExecutor-3369-thread-1-processing-n:127.0.0.1:60979_) [n:127.0.0.1:60979_ c:collection1   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1925558 INFO  (coreLoadExecutor-3369-thread-1-processing-n:127.0.0.1:60979_) [n:127.0.0.1:60979_ c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1925809 INFO  (coreLoadExecutor-3369-thread-1-processing-n:127.0.0.1:60979_) [n:127.0.0.1:60979_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1925817 INFO  (coreLoadExecutor-3369-thread-1-processing-n:127.0.0.1:60979_) [n:127.0.0.1:60979_ c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1925819 INFO  (coreLoadExecutor-3369-thread-1-processing-n:127.0.0.1:60979_) [n:127.0.0.1:60979_ c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1925828 INFO  (coreLoadExecutor-3369-thread-1-processing-n:127.0.0.1:60979_) [n:127.0.0.1:60979_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1925831 INFO  (coreLoadExecutor-3369-thread-1-processing-n:127.0.0.1:60979_) [n:127.0.0.1:60979_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1925833 INFO  (coreLoadExecutor-3369-thread-1-processing-n:127.0.0.1:60979_) [n:127.0.0.1:60979_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1925834 INFO  (coreLoadExecutor-3369-thread-1-processing-n:127.0.0.1:60979_) [n:127.0.0.1:60979_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1925836 INFO  (coreLoadExecutor-3369-thread-1-processing-n:127.0.0.1:60979_) [n:127.0.0.1:60979_ c:collection1   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1925836 INFO  (coreLoadExecutor-3369-thread-1-processing-n:127.0.0.1:60979_) [n:127.0.0.1:60979_ c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]  

[...truncated too long message...]

ins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_FD3FB35516E95D91-001/solr-instance-026/./collection1/data
   [junit4]   2> 2840112 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[FD3FB35516E95D91]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_FD3FB35516E95D91-001/solr-instance-026/./collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_FD3FB35516E95D91-001/solr-instance-026/./collection1/data/index;done=false>>]
   [junit4]   2> 2840112 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[FD3FB35516E95D91]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_FD3FB35516E95D91-001/solr-instance-026/./collection1/data/index
   [junit4]   2> 2840115 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[FD3FB35516E95D91]) [    x:collection1] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@712cfc94{/solr,null,UNAVAILABLE}
   [junit4]   2> 2840117 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[FD3FB35516E95D91]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@4193d246{HTTP/1.1}{127.0.0.1:0}
   [junit4]   2> 2840117 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[FD3FB35516E95D91]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1468793681
   [junit4]   2> 2840117 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[FD3FB35516E95D91]) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@7c03a5c9
   [junit4]   2> 2840117 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[FD3FB35516E95D91]) [    x:collection1] o.a.s.u.DirectUpdateHandler2 closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 2840118 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[FD3FB35516E95D91]) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 2840118 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[FD3FB35516E95D91]) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 2840118 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[FD3FB35516E95D91]) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 2840118 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[FD3FB35516E95D91]) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 2840119 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[FD3FB35516E95D91]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 2840119 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[FD3FB35516E95D91]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_FD3FB35516E95D91-001/solr-instance-027/./collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_FD3FB35516E95D91-001/solr-instance-027/./collection1/data;done=false>>]
   [junit4]   2> 2840119 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[FD3FB35516E95D91]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_FD3FB35516E95D91-001/solr-instance-027/./collection1/data
   [junit4]   2> 2840119 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[FD3FB35516E95D91]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_FD3FB35516E95D91-001/solr-instance-027/./collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_FD3FB35516E95D91-001/solr-instance-027/./collection1/data/index;done=false>>]
   [junit4]   2> 2840119 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[FD3FB35516E95D91]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_FD3FB35516E95D91-001/solr-instance-027/./collection1/data/index
   [junit4]   2> 2840122 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[FD3FB35516E95D91]) [    x:collection1] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3b0758e3{/solr,null,UNAVAILABLE}
   [junit4]   2> 2840123 INFO  (SUITE-TestReplicationHandler-seed#[FD3FB35516E95D91]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_FD3FB35516E95D91-001
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=bg, timezone=Etc/GMT0
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=128172608,total=504889344
   [junit4]   2> NOTE: All tests run in this JVM: [SuggestComponentTest, HighlighterTest, FastVectorHighlighterTest, DistributedFacetPivotSmallAdvancedTest, TestHashPartitioner, TestRuleBasedAuthorizationPlugin, TestBulkSchemaAPI, DistributedIntervalFacetingTest, TestSolrConfigHandler, TestSchemaResource, CollectionTooManyReplicasTest, SecurityConfHandlerTest, SuggesterFSTTest, TestStressVersions, TestQuerySenderListener, TestUniqueKeyFieldResource, CloudExitableDirectoryReaderTest, ChaosMonkeyNothingIsSafeTest, DistributedMLTComponentTest, ShardRoutingCustomTest, DirectoryFactoryTest, DocExpirationUpdateProcessorFactoryTest, TestClusterStateMutator, SolrIndexConfigTest, SolrCloudExampleTest, TestLFUCache, SpellingQueryConverterTest, TestQueryTypes, TestOrdValues, TestPivotHelperCode, OpenCloseCoreStressTest, TestCloudInspectUtil, DistributedQueryComponentCustomSortTest, ZkCLITest, CloudMLTQParserTest, TestDefaultStatsCache, SolrPluginUtilsTest, UniqFieldsUpdateProcessorFactoryTest, VMParamsZkACLAndCredentialsProvidersTest, TestSimpleTrackingShardHandler, SmileWriterTest, TestReplicaProperties, TestMissingGroups, TestChildDocTransformer, DistributedExpandComponentTest, DeleteShardTest, SpellCheckCollatorTest, SortByFunctionTest, ZkControllerTest, TestSimpleQParserPlugin, HighlighterMaxOffsetTest, DocValuesTest, OverseerCollectionConfigSetProcessorTest, TestBM25SimilarityFactory, SpatialFilterTest, TestPartialUpdateDeduplication, ClusterStateUpdateTest, DistanceUnitsTest, TestMiniSolrCloudCluster, TestCoreContainer, TestReplicationHandlerBackup, TestTrie, TestFieldTypeResource, OverseerTaskQueueTest, BlockJoinFacetDistribTest, TestLMJelinekMercerSimilarityFactory, TestCodecSupport, ConvertedLegacyTest, TestExclusionRuleCollectionAccess, TestExceedMaxTermLength, TestConfigSetsAPI, TestAddFieldRealTimeGet, IndexBasedSpellCheckerTest, TestSolrCoreProperties, TestFileDictionaryLookup, WordBreakSolrSpellCheckerTest, TestConfigSetProperties, TestManagedSchemaFieldTypeResource, TestDistributedGrouping, TestShortCircuitedRequests, MinimalSchemaTest, TestXmlQParser, ForceLeaderTest, PreAnalyzedUpdateProcessorTest, NotRequiredUniqueKeyTest, TestHashQParserPlugin, DistanceFunctionTest, TestSchemaManager, TestSerializedLuceneMatchVersion, RequestLoggingTest, BasicDistributedZkTest, ChaosMonkeySafeLeaderTest, CollectionsAPIDistributedZkTest, SyncSliceTest, OverseerTest, ShardRoutingTest, RecoveryZkTest, TestReplicationHandler]
   [junit4] Completed [202/565 (2!)] on J1 in 424.79s, 13 tests, 1 failure <<< FAILURES!

[...truncated 1269 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:534: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1444: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1000: There were test failures: 565 suites (7 ignored), 2259 tests, 1 error, 1 failure, 33 ignored (18 assumptions) [seed: FD3FB35516E95D91]

Total time: 260 minutes 10 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1075 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1075/

1 tests failed.
FAILED:  org.apache.solr.cloud.BasicDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:37382/collection1: Bad Request    request: http://127.0.0.1:41246/collection1/update?update.chain=distrib-dup-test-chain-explicit&update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A37382%2Fcollection1%2F&wt=javabin&version=2

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37382/collection1: Bad Request



request: http://127.0.0.1:41246/collection1/update?update.chain=distrib-dup-test-chain-explicit&update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A37382%2Fcollection1%2F&wt=javabin&version=2
	at __randomizedtesting.SeedInfo.seed([1817E361ACAC99E4:9043DCBB0250F41C]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:576)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:167)
	at org.apache.solr.BaseDistributedSearchTestCase.add(BaseDistributedSearchTestCase.java:512)
	at org.apache.solr.cloud.BasicDistributedZkTest.testUpdateProcessorsRunOnlyOnce(BasicDistributedZkTest.java:653)
	at org.apache.solr.cloud.BasicDistributedZkTest.test(BasicDistributedZkTest.java:364)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:965)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:940)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11068 lines...]
   [junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/init-core-data-001
   [junit4]   2> 1884484 INFO  (SUITE-BasicDistributedZkTest-seed#[1817E361ACAC99E4]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 1884489 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1884489 INFO  (Thread-3374) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1884489 INFO  (Thread-3374) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1884589 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.ZkTestServer start zk server on port:39969
   [junit4]   2> 1884590 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1884590 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1884594 INFO  (zkCallback-1754-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@55568b2 name:ZooKeeperConnection Watcher:127.0.0.1:39969 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1884594 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1884595 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1884595 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1884598 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1884599 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1884603 INFO  (zkCallback-1755-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@438b8942 name:ZooKeeperConnection Watcher:127.0.0.1:39969/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1884603 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1884604 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1884604 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1884607 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1884609 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1884611 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1884612 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1884612 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1884616 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1884616 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1884618 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1884618 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1884619 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1884620 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1884621 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1884621 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1884623 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1884623 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1884624 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1884624 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1884626 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1884626 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1884627 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1884627 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1884629 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1884629 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1884630 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1884631 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1885134 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/control-001/cores/collection1
   [junit4]   2> 1885136 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1885139 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4636109d{/,null,AVAILABLE}
   [junit4]   2> 1885139 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.e.j.s.ServerConnector Started ServerConnector@15237a99{HTTP/1.1}{127.0.0.1:60904}
   [junit4]   2> 1885139 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.e.j.s.Server Started @1888870ms
   [junit4]   2> 1885139 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=60904, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/control-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/tempDir-001/control/data, hostContext=/}
   [junit4]   2> 1885140 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1885140 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/control-001'
   [junit4]   2> 1885140 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1885140 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1885140 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1885141 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1885144 INFO  (zkCallback-1756-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@207a3426 name:ZooKeeperConnection Watcher:127.0.0.1:39969/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1885144 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1885144 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1885146 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1885146 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/control-001/solr.xml
   [junit4]   2> 1885162 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/control-001/cores
   [junit4]   2> 1885162 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.CoreContainer New CoreContainer 202555699
   [junit4]   2> 1885162 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/control-001]
   [junit4]   2> 1885162 WARN  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/control-001/lib
   [junit4]   2> 1885162 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1885163 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1885164 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1885164 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1885164 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39969/solr
   [junit4]   2> 1885164 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1885164 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1885165 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1885167 INFO  (zkCallback-1758-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@75f1b685 name:ZooKeeperConnection Watcher:127.0.0.1:39969 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1885167 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1885167 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1885169 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:60904_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1885171 INFO  (zkCallback-1759-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@41581898 name:ZooKeeperConnection Watcher:127.0.0.1:39969/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1885172 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:60904_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1885174 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:60904_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1885177 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:60904_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1885180 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:60904_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1885183 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:60904_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1885187 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:60904_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1885190 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:60904_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1885192 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:60904_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1885194 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:60904_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1885195 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:60904_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1885196 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:60904_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60904_
   [junit4]   2> 1885197 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:60904_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:60904_
   [junit4]   2> 1885199 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:60904_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1885200 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:60904_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1885201 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:60904_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1885202 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:60904_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95213991123484676-127.0.0.1:60904_-n_0000000000
   [junit4]   2> 1885202 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:60904_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:60904_
   [junit4]   2> 1885202 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:60904_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1885204 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:60904_    ] o.a.s.c.Overseer Overseer (id=95213991123484676-127.0.0.1:60904_-n_0000000000) starting
   [junit4]   2> 1885206 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:60904_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1885213 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:60904_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1885214 INFO  (OverseerCollectionConfigSetProcessor-95213991123484676-127.0.0.1:60904_-n_0000000000) [n:127.0.0.1:60904_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1885214 INFO  (OverseerStateUpdate-95213991123484676-127.0.0.1:60904_-n_0000000000) [n:127.0.0.1:60904_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1885214 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:60904_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1885241 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:60904_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1885241 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:60904_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1885242 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:60904_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/control-001/cores
   [junit4]   2> 1885243 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:60904_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {coreNodeName=, shard=, collection=control_collection, loadOnStartup=true, dataDir=data/, config=solrconfig.xml, configSetProperties=configsetprops.json, schema=schema.xml, transient=false, name=collection1}
   [junit4]   2> 1885243 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:60904_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/control-001/cores/collection1
   [junit4]   2> 1885243 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:60904_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1885245 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:60904_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1885245 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:60904_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1885245 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1885245 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1885247 INFO  (OverseerStateUpdate-95213991123484676-127.0.0.1:60904_-n_0000000000) [n:127.0.0.1:60904_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:60904_",
   [junit4]   2>   "base_url":"http://127.0.0.1:60904"} current state version: 0
   [junit4]   2> 1885248 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1885248 INFO  (OverseerStateUpdate-95213991123484676-127.0.0.1:60904_-n_0000000000) [n:127.0.0.1:60904_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:60904_",
   [junit4]   2>   "base_url":"http://127.0.0.1:60904"}
   [junit4]   2> 1885249 INFO  (OverseerStateUpdate-95213991123484676-127.0.0.1:60904_-n_0000000000) [n:127.0.0.1:60904_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1885249 INFO  (OverseerStateUpdate-95213991123484676-127.0.0.1:60904_-n_0000000000) [n:127.0.0.1:60904_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1885251 INFO  (zkCallback-1759-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1886249 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1886249 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1886249 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1886249 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 1886250 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1886250 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/control-001/cores/collection1'
   [junit4]   2> 1886250 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1886250 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1886252 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1886258 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1886272 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1886301 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1886314 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1886601 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1886610 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1886612 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1886628 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1886632 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1886637 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1886638 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1886640 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1886640 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1886640 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1886640 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/control-001/cores/collection1/data/]
   [junit4]   2> 1886640 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3ca63627
   [junit4]   2> 1886641 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/control-001/cores/collection1/data
   [junit4]   2> 1886641 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1886641 WARN  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1886642 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/control-001/cores/collection1/data/index
   [junit4]   2> 1886642 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@2768ffa4
   [junit4]   2> 1886645 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(NIOFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/index-NIOFSDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5cc4aaec),segFN=segments_1,generation=1}
   [junit4]   2> 1886646 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1886666 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1886666 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1886667 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1886667 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1886667 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1886667 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1886667 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1886667 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1886667 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1886668 WARN  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 1886671 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1886671 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/control-001/cores/collection1/data/
   [junit4]   2> 1886672 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1886672 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1886674 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1886677 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1886683 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/segments,/get,/admin/luke,standard,/admin/ping,/admin/threads,/dump,/update/json,/admin/properties,/admin/mbeans,/admin/logging,/admin/plugins,/config,/update/csv,/schema,/update/json/docs,/update,/replication,/admin/file,/admin/system
   [junit4]   2> 1886686 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1886686 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1886686 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1886687 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1886688 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1886688 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=38, maxMergeAtOnceExplicit=46, maxMergedSegmentMB=84.8212890625, floorSegmentMB=1.1318359375, forceMergeDeletesPctAllowed=27.90915906757699, segmentsPerTier=43.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1886689 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(NIOFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/index-NIOFSDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5cc4aaec),segFN=segments_1,generation=1}
   [junit4]   2> 1886689 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1886690 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@343a656c[collection1] main
   [junit4]   2> 1886690 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 1886697 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1886697 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1886698 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1886698 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1886698 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1886700 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1886700 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1886700 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1886700 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1886702 INFO  (searcherExecutor-3391-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@343a656c[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1886702 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1886702 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1886702 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1886702 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1523423860294156288
   [junit4]   2> 1886705 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 3.0ms to seed version buckets with highest version 1523423860294156288
   [junit4]   2> 1886705 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1886706 INFO  (coreLoadExecutor-3390-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1886707 INFO  (coreZkRegister-3384-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:60904 collection:control_collection shard:shard1
   [junit4]   2> 1886708 INFO  (coreZkRegister-3384-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1886712 INFO  (coreZkRegister-3384-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95213991123484676-core_node1-n_0000000000
   [junit4]   2> 1886713 INFO  (coreZkRegister-3384-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 1886715 INFO  (coreZkRegister-3384-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1886715 INFO  (coreZkRegister-3384-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1886715 INFO  (coreZkRegister-3384-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:60904/collection1/
   [junit4]   2> 1886715 INFO  (OverseerStateUpdate-95213991123484676-127.0.0.1:60904_-n_0000000000) [n:127.0.0.1:60904_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1886715 INFO  (coreZkRegister-3384-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1886716 INFO  (coreZkRegister-3384-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:60904/collection1/ has no replicas
   [junit4]   2> 1886717 INFO  (coreZkRegister-3384-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1886720 INFO  (coreZkRegister-3384-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1886722 INFO  (coreZkRegister-3384-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:60904/collection1/ shard1
   [junit4]   2> 1886723 INFO  (OverseerStateUpdate-95213991123484676-127.0.0.1:60904_-n_0000000000) [n:127.0.0.1:60904_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:60904",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1886747 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1886749 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1886749 INFO  (zkCallback-1761-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@20eded13 name:ZooKeeperConnection Watcher:127.0.0.1:39969/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1886749 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1886749 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1886749 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1886751 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1886826 INFO  (zkCallback-1761-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1886826 INFO  (zkCallback-1759-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1886874 INFO  (coreZkRegister-3384-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:60904/collection1/ and leader is http://127.0.0.1:60904/collection1/
   [junit4]   2> 1886874 INFO  (coreZkRegister-3384-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:60904
   [junit4]   2> 1886874 INFO  (coreZkRegister-3384-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1886874 INFO  (coreZkRegister-3384-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1886874 INFO  (coreZkRegister-3384-thread-1-processing-r:core_node1 s:shard1 c:control_collection x:collection1 n:127.0.0.1:60904_) [n:127.0.0.1:60904_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1886876 INFO  (OverseerStateUpdate-95213991123484676-127.0.0.1:60904_-n_0000000000) [n:127.0.0.1:60904_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:60904_",
   [junit4]   2>   "base_url":"http://127.0.0.1:60904"} current state version: 2
   [junit4]   2> 1886877 INFO  (OverseerStateUpdate-95213991123484676-127.0.0.1:60904_-n_0000000000) [n:127.0.0.1:60904_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:60904_",
   [junit4]   2>   "base_url":"http://127.0.0.1:60904"}
   [junit4]   2> 1886980 INFO  (zkCallback-1759-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1886980 INFO  (zkCallback-1761-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1887142 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/shard-1-001/cores/collection1
   [junit4]   2> 1887144 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/shard-1-001
   [junit4]   2> 1887153 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1887177 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@47fb729{/,null,AVAILABLE}
   [junit4]   2> 1887177 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.e.j.s.ServerConnector Started ServerConnector@5cd1e70d{HTTP/1.1}{127.0.0.1:37382}
   [junit4]   2> 1887178 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.e.j.s.Server Started @1890909ms
   [junit4]   2> 1887178 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, hostPort=37382, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/shard-1-001/cores, hostContext=/, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/tempDir-001/jetty1}
   [junit4]   2> 1887178 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1887178 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/shard-1-001'
   [junit4]   2> 1887178 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1887178 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1887179 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1887183 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1887184 INFO  (zkCallback-1762-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@18fcabbb name:ZooKeeperConnection Watcher:127.0.0.1:39969/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1887184 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1887184 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1887185 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1887185 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/shard-1-001/solr.xml
   [junit4]   2> 1887209 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/shard-1-001/cores
   [junit4]   2> 1887210 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.CoreContainer New CoreContainer 1974690352
   [junit4]   2> 1887210 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/shard-1-001]
   [junit4]   2> 1887210 WARN  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/shard-1-001/lib
   [junit4]   2> 1887210 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1887212 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1887213 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1887213 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1887213 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39969/solr
   [junit4]   2> 1887215 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1887215 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1887251 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1887290 INFO  (zkCallback-1764-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1da124d5 name:ZooKeeperConnection Watcher:127.0.0.1:39969 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1887290 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1887291 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1887293 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:37382_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1887295 INFO  (zkCallback-1765-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@34b71c5c name:ZooKeeperConnection Watcher:127.0.0.1:39969/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1887295 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:37382_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1887301 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:37382_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1888305 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:37382_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37382_
   [junit4]   2> 1888306 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:37382_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:37382_
   [junit4]   2> 1888307 INFO  (zkCallback-1759-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1888307 INFO  (zkCallback-1761-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1888308 INFO  (zkCallback-1765-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1888308 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:37382_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1888309 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:37382_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95213991123484680-127.0.0.1:37382_-n_0000000001
   [junit4]   2> 1888309 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:37382_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/95213991123484676-127.0.0.1:60904_-n_0000000000 to know if I could be the leader
   [junit4]   2> 1888330 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:37382_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1888330 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:37382_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1888331 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:37382_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/shard-1-001/cores
   [junit4]   2> 1888332 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:37382_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, coreNodeName=, dataDir=data/, collection=collection1, configSetProperties=configsetprops.json, shard=, schema=schema.xml, transient=false, config=solrconfig.xml, loadOnStartup=true}
   [junit4]   2> 1888332 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:37382_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/shard-1-001/cores/collection1
   [junit4]   2> 1888332 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:37382_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1888334 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1888334 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1888336 INFO  (OverseerStateUpdate-95213991123484676-127.0.0.1:60904_-n_0000000000) [n:127.0.0.1:60904_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:37382_",
   [junit4]   2>   "base_url":"http://127.0.0.1:37382"} current state version: 3
   [junit4]   2> 1888336 INFO  (OverseerStateUpdate-95213991123484676-127.0.0.1:60904_-n_0000000000) [n:127.0.0.1:60904_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:37382_",
   [junit4]   2>   "base_url":"http://127.0.0.1:37382"}
   [junit4]   2> 1888336 INFO  (OverseerStateUpdate-95213991123484676-127.0.0.1:60904_-n_0000000000) [n:127.0.0.1:60904_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1888336 INFO  (OverseerStateUpdate-95213991123484676-127.0.0.1:60904_-n_0000000000) [n:127.0.0.1:60904_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 1888338 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1888339 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:37382_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1888339 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:37382_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1888441 INFO  (zkCallback-1761-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1888441 INFO  (zkCallback-1765-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1888441 INFO  (zkCallback-1759-thread-1-processing-n:127.0.0.1:60904_) [n:127.0.0.1:60904_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1889339 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1889339 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1889340 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1889340 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/collection1]
   [junit4]   2> 1889340 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/collection1] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1889340 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/shard-1-001/cores/collection1'
   [junit4]   2> 1889340 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1889341 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1889342 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1889348 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1889362 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1889391 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1889399 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1889652 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1889660 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1889662 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1889671 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1889674 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1889676 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1889677 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1889679 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1889679 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 1889679 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1889679 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/shard-1-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/shard-1-001/cores/collection1/data/]
   [junit4]   2> 1889679 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3ca63627
   [junit4]   2> 1889680 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/shard-1-001/cores/collection1/data
   [junit4]   2> 1889680 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/shard-1-001/cores/collection1/data/index/
   [junit4]   2> 1889680 WARN  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/shard-1-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1889681 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/shard-1-001/cores/collection1/data/index
   [junit4]   2> 1889681 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@107bc95c
   [junit4]   2> 1889684 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(NIOFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/index-NIOFSDirectory-002 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5cc4aaec),segFN=segments_1,generation=1}
   [junit4]   2> 1889684 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1889685 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:37382_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/shard-1-001/cores/collection1/data/
   [junit4]   2> 1889685 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:37382_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/shard-1-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1889690 INFO  (coreLoadExecutor-3401-thread-1-processing-n:127.0.0.1:37382_) [n:127.0.0.1:37382_ c:collec

[...truncated too long message...]

 o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001/shard-4-001/cores/collection1/data/index
   [junit4]   2> 1963576 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@6893394c{HTTP/1.1}{127.0.0.1:0}
   [junit4]   2> 1963576 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@150c8397{/,null,UNAVAILABLE}
   [junit4]   2> 1963578 INFO  (TEST-BasicDistributedZkTest.test-seed#[1817E361ACAC99E4]) [n:127.0.0.1:47355_ c:collection3 s:shard1 r:core_node4 x:collection3] o.a.s.c.ZkTestServer connecting to 127.0.0.1:39969 39969
   [junit4]   2> 1963698 INFO  (Thread-3374) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:39969 39969
   [junit4]   2> 1963699 WARN  (Thread-3374) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	7	/solr/aliases.json
   [junit4]   2> 	5	/solr/configs/conf1
   [junit4]   2> 	5	/solr/security.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	7	/solr/clusterstate.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	7	/solr/live_nodes
   [junit4]   2> 	7	/solr/collections
   [junit4]   2> 	4	/solr/overseer/collection-queue-work
   [junit4]   2> 	4	/solr/overseer/queue
   [junit4]   2> 	4	/solr/overseer/queue-work
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=BasicDistributedZkTest -Dtests.method=test -Dtests.seed=1817E361ACAC99E4 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=fr_CH -Dtests.timezone=Asia/Nicosia -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   79.2s J0 | BasicDistributedZkTest.test <<<
   [junit4]    > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37382/collection1: Bad Request
   [junit4]    > request: http://127.0.0.1:41246/collection1/update?update.chain=distrib-dup-test-chain-explicit&update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A37382%2Fcollection1%2F&wt=javabin&version=2
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([1817E361ACAC99E4:9043DCBB0250F41C]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:576)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:167)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.add(BaseDistributedSearchTestCase.java:512)
   [junit4]    > 	at org.apache.solr.cloud.BasicDistributedZkTest.testUpdateProcessorsRunOnlyOnce(BasicDistributedZkTest.java:653)
   [junit4]    > 	at org.apache.solr.cloud.BasicDistributedZkTest.test(BasicDistributedZkTest.java:364)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:965)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:940)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 1963704 INFO  (SUITE-BasicDistributedZkTest-seed#[1817E361ACAC99E4]-worker) [n:127.0.0.1:47355_ c:collection3 s:shard1 r:core_node4 x:collection3] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_1817E361ACAC99E4-001
   [junit4]   2> Jan 15, 2016 9:31:25 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {n_d1=PostingsFormat(name=LuceneFixedGap), intDefault=Lucene50(blocksize=128), foo_b=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), regex_dup_B_s=PostingsFormat(name=LuceneFixedGap), n_l1=PostingsFormat(name=LuceneFixedGap), author_t=Lucene50(blocksize=128), text=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), genre_s=BlockTreeOrds(blocksize=128), price=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), regex_dup_A_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), foo_i=Lucene50(blocksize=128), a_t=PostingsFormat(name=LuceneFixedGap), SubjectTerms_mfacet=Lucene50(blocksize=128), foo_f=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), n_td1=Lucene50(blocksize=128), id=BlockTreeOrds(blocksize=128), n_f1=Lucene50(blocksize=128), sequence_i=BlockTreeOrds(blocksize=128), oddField_s=BlockTreeOrds(blocksize=128), range_facet_i_dv=PostingsFormat(name=LuceneFixedGap), n_tf1=PostingsFormat(name=LuceneFixedGap), inStock=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), val_i=BlockTreeOrds(blocksize=128), name=BlockTreeOrds(blocksize=128), rnd_b=BlockTreeOrds(blocksize=128), other_tl1=Lucene50(blocksize=128), n_tdt1=Lucene50(blocksize=128), range_facet_l_dv=BlockTreeOrds(blocksize=128), timestamp=PostingsFormat(name=LuceneFixedGap), cat=PostingsFormat(name=LuceneFixedGap), a_i1=PostingsFormat(name=LuceneFixedGap), foo_d=BlockTreeOrds(blocksize=128), n_dt1=Lucene50(blocksize=128), series_t=Lucene50(blocksize=128), range_facet_l=PostingsFormat(name=LuceneFixedGap), n_tl1=Lucene50(blocksize=128), multiDefault=PostingsFormat(name=LuceneFixedGap), _version_=Lucene50(blocksize=128), n_ti1=BlockTreeOrds(blocksize=128)}, docValues:{timestamp=DocValuesFormat(name=Direct), range_facet_i_dv=DocValuesFormat(name=Direct), range_facet_l_dv=DocValuesFormat(name=Lucene54)}, sim=DefaultSimilarity, locale=fr_CH, timezone=Asia/Nicosia
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=291418576,total=501743616
   [junit4]   2> NOTE: All tests run in this JVM: [TestFunctionQuery, SortSpecParsingTest, TestRandomFaceting, PolyFieldTest, TestManagedSynonymFilterFactory, BlockJoinFacetSimpleTest, TestSolr4Spatial2, ParsingFieldUpdateProcessorsTest, TestSchemaNameResource, TestLuceneMatchVersion, TestLazyCores, DistributedVersionInfoTest, TestFieldTypeCollectionResource, TestMaxScoreQueryParser, TestConfigSetsAPIZkFailure, SyncSliceTest, TestRebalanceLeaders, TestRTGBase, TestRandomDVFaceting, SpatialRPTFieldTypeTest, JavabinLoaderTest, TestRTimerTree, DistributedDebugComponentTest, CleanupOldIndexTest, ResourceLoaderTest, SOLR749Test, AlternateDirectoryTest, TestAtomicUpdateErrorCases, TestFaceting, ReturnFieldsTest, TestJsonRequest, ActionThrottleTest, TestManagedSchema, TestShardHandlerFactory, BufferStoreTest, SignatureUpdateProcessorFactoryTest, ShardSplitTest, TriLevelCompositeIdRoutingTest, TestJoin, TestIBSimilarityFactory, PreAnalyzedFieldTest, EchoParamsTest, DistributedQueryElevationComponentTest, TestInitParams, LeaderFailoverAfterPartitionTest, TestSolrCLIRunExample, HdfsDirectoryFactoryTest, CurrencyFieldOpenExchangeTest, BasicAuthIntegrationTest, DistributedFacetPivotWhiteBoxTest, TestLRUCache, TestCloudManagedSchemaConcurrent, TestRemoteStreaming, TestRawResponseWriter, DocumentAnalysisRequestHandlerTest, TestMacros, TestFoldingMultitermQuery, TestTrieFacet, TestStressLucene, JsonLoaderTest, TestAuthenticationFramework, LeaderElectionIntegrationTest, TestDynamicLoading, TestHighFrequencyDictionaryFactory, TestSolrXml, TestMinMaxOnMultiValuedField, QueryEqualityTest, BasicDistributedZkTest]
   [junit4] Completed [256/565 (1!)] on J0 in 79.76s, 1 test, 1 error <<< FAILURES!

[...truncated 998 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:534: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1444: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1000: There were test failures: 565 suites (7 ignored), 2258 tests, 1 error, 33 ignored (18 assumptions) [seed: 1817E361ACAC99E4]

Total time: 221 minutes 58 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1074 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1074/

1 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=33805, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=33805, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:51792/i_q/p: collection already exists: awholynewstresscollection_collection1_5
	at __randomizedtesting.SeedInfo.seed([95EB1EF282912922]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:881)




Build Log:
[...truncated 11194 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/init-core-data-001
   [junit4]   2> 1963273 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[95EB1EF282912922]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false)
   [junit4]   2> 1963273 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[95EB1EF282912922]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /i_q/p
   [junit4]   2> 1963278 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1963278 INFO  (Thread-27858) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1963278 INFO  (Thread-27858) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1963378 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.ZkTestServer start zk server on port:56217
   [junit4]   2> 1963378 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1963379 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1963390 INFO  (zkCallback-909-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@17989f08 name:ZooKeeperConnection Watcher:127.0.0.1:56217 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1963390 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1963390 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1963390 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1963394 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1963394 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1963396 INFO  (zkCallback-910-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@53668b15 name:ZooKeeperConnection Watcher:127.0.0.1:56217/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1963396 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1963396 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1963396 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1963399 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1963401 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1963402 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1963405 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1963405 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1963409 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1963409 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1963412 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1963413 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1963415 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1963415 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1963418 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1963418 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1963422 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1963422 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1963427 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1963427 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1963429 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1963430 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1963437 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1963437 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1963443 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1963443 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1963447 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1963447 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1963462 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1963463 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1963466 INFO  (zkCallback-911-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4fca1940 name:ZooKeeperConnection Watcher:127.0.0.1:56217/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1963466 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1963466 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1963930 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/control-001/cores/collection1
   [junit4]   2> 1963931 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1963947 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@88864d6{/i_q/p,null,AVAILABLE}
   [junit4]   2> 1963953 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.e.j.s.ServerConnector Started ServerConnector@3ac9cb8c{SSL-http/1.1}{127.0.0.1:55461}
   [junit4]   2> 1963953 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.e.j.s.Server Started @1966606ms
   [junit4]   2> 1963953 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/i_q/p, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/tempDir-001/control/data, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/control-001/cores, hostPort=55461}
   [junit4]   2> 1963954 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1963954 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/control-001'
   [junit4]   2> 1963954 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1963954 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1963954 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1963955 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1963957 INFO  (zkCallback-912-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7561d912 name:ZooKeeperConnection Watcher:127.0.0.1:56217/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1963957 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1963958 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1963964 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1963964 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/control-001/solr.xml
   [junit4]   2> 1963981 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/control-001/cores
   [junit4]   2> 1963981 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.CoreContainer New CoreContainer 1846390021
   [junit4]   2> 1963981 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/control-001]
   [junit4]   2> 1963982 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/control-001/lib
   [junit4]   2> 1963983 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1963986 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1963987 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1963987 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1963988 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56217/solr
   [junit4]   2> 1963988 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1963988 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1963988 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1963990 INFO  (zkCallback-914-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@12bbc73c name:ZooKeeperConnection Watcher:127.0.0.1:56217 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1963990 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1963991 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1963995 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1963998 INFO  (zkCallback-915-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2b2c03d0 name:ZooKeeperConnection Watcher:127.0.0.1:56217/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1963998 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1964000 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1964007 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1964011 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1964014 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1964021 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1964025 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1964029 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1964031 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1964032 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1964033 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55461_i_q%2Fp
   [junit4]   2> 1964034 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:55461_i_q%2Fp
   [junit4]   2> 1964037 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1964039 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1964042 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1964043 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95203506598772741-127.0.0.1:55461_i_q%2Fp-n_0000000000
   [junit4]   2> 1964043 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:55461_i_q%2Fp
   [junit4]   2> 1964043 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1964047 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.Overseer Overseer (id=95203506598772741-127.0.0.1:55461_i_q%2Fp-n_0000000000) starting
   [junit4]   2> 1964050 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1964062 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1964063 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1964063 INFO  (OverseerCollectionConfigSetProcessor-95203506598772741-127.0.0.1:55461_i_q%2Fp-n_0000000000) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1964064 INFO  (OverseerStateUpdate-95203506598772741-127.0.0.1:55461_i_q%2Fp-n_0000000000) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1964087 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1964087 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1964089 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/control-001/cores
   [junit4]   2> 1964090 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {coreNodeName=, collection=control_collection, loadOnStartup=true, transient=false, dataDir=data/, schema=schema.xml, configSetProperties=configsetprops.json, config=solrconfig.xml, shard=, name=collection1}
   [junit4]   2> 1964090 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/control-001/cores/collection1
   [junit4]   2> 1964090 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1964093 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1964093 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1964096 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1964096 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1964097 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1964098 INFO  (OverseerStateUpdate-95203506598772741-127.0.0.1:55461_i_q%2Fp-n_0000000000) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:55461/i_q/p",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:55461_i_q%2Fp",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1"} current state version: 0
   [junit4]   2> 1964100 INFO  (OverseerStateUpdate-95203506598772741-127.0.0.1:55461_i_q%2Fp-n_0000000000) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:55461/i_q/p",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:55461_i_q%2Fp",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1"}
   [junit4]   2> 1964101 INFO  (OverseerStateUpdate-95203506598772741-127.0.0.1:55461_i_q%2Fp-n_0000000000) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1964101 INFO  (OverseerStateUpdate-95203506598772741-127.0.0.1:55461_i_q%2Fp-n_0000000000) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1964104 INFO  (zkCallback-915-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1965098 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1965098 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1965099 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1965099 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 1965100 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1965100 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/control-001/cores/collection1'
   [junit4]   2> 1965100 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1965100 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1965103 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1965112 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1965143 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1965173 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1965181 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1965499 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1965519 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1965521 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1965539 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1965543 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1965546 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1965547 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1965549 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1965549 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1965549 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1965549 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/control-001/cores/collection1/data/]
   [junit4]   2> 1965550 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@385e0377
   [junit4]   2> 1965550 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/control-001/cores/collection1/data
   [junit4]   2> 1965550 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1965551 WARN  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1965551 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/control-001/cores/collection1/data/index
   [junit4]   2> 1965551 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=49, maxMergeAtOnceExplicit=25, maxMergedSegmentMB=0.9697265625, floorSegmentMB=0.720703125, forceMergeDeletesPctAllowed=20.70119545137434, segmentsPerTier=47.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.850076328410624
   [junit4]   2> 1965554 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/index-SimpleFSDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ba52c2a),segFN=segments_1,generation=1}
   [junit4]   2> 1965555 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1965559 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1965559 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1965560 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1965560 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1965560 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1965560 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1965560 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1965560 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1965560 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1965560 WARN  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 1965562 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/control-001/cores/collection1/data/
   [junit4]   2> 1965563 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1965564 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1965565 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1965567 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1965568 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1965572 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/threads,/update/json,/admin/luke,/replication,/dump,/get,standard,/admin/mbeans,/update/json/docs,/admin/properties,/admin/logging,/config,/admin/system,/admin/plugins,/update,/admin/file,/update/csv,/admin/segments,/admin/ping,/schema
   [junit4]   2> 1965573 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1965574 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1965574 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1965574 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1965574 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1965575 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@1ee0bedb
   [junit4]   2> 1965576 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/index-SimpleFSDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ba52c2a),segFN=segments_1,generation=1}
   [junit4]   2> 1965576 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1965576 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@7abd564c[collection1] main
   [junit4]   2> 1965577 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 1965578 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1965578 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1965578 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1965578 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1965578 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1965579 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1965579 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1965579 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1965579 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1965579 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1965579 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1965579 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1965579 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1523256107992088576
   [junit4]   2> 1965580 INFO  (searcherExecutor-2631-thread-1-processing-c:control_collection s:shard1 x:collection1 n:127.0.0.1:55461_i_q%2Fp r:core_node1) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7abd564c[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1965582 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1523256107992088576
   [junit4]   2> 1965582 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1965583 INFO  (coreLoadExecutor-2630-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1965583 INFO  (coreZkRegister-2624-thread-1-processing-c:control_collection s:shard1 x:collection1 n:127.0.0.1:55461_i_q%2Fp r:core_node1) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:https://127.0.0.1:55461/i_q/p collection:control_collection shard:shard1
   [junit4]   2> 1965584 INFO  (coreZkRegister-2624-thread-1-processing-c:control_collection s:shard1 x:collection1 n:127.0.0.1:55461_i_q%2Fp r:core_node1) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1965588 INFO  (coreZkRegister-2624-thread-1-processing-c:control_collection s:shard1 x:collection1 n:127.0.0.1:55461_i_q%2Fp r:core_node1) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95203506598772741-core_node1-n_0000000000
   [junit4]   2> 1965588 INFO  (coreZkRegister-2624-thread-1-processing-c:control_collection s:shard1 x:collection1 n:127.0.0.1:55461_i_q%2Fp r:core_node1) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=180000
   [junit4]   2> 1965591 INFO  (OverseerStateUpdate-95203506598772741-127.0.0.1:55461_i_q%2Fp-n_0000000000) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1965593 INFO  (coreZkRegister-2624-thread-1-processing-c:control_collection s:shard1 x:collection1 n:127.0.0.1:55461_i_q%2Fp r:core_node1) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1965593 INFO  (coreZkRegister-2624-thread-1-processing-c:control_collection s:shard1 x:collection1 n:127.0.0.1:55461_i_q%2Fp r:core_node1) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1965593 INFO  (coreZkRegister-2624-thread-1-processing-c:control_collection s:shard1 x:collection1 n:127.0.0.1:55461_i_q%2Fp r:core_node1) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:55461/i_q/p/collection1/
   [junit4]   2> 1965593 INFO  (coreZkRegister-2624-thread-1-processing-c:control_collection s:shard1 x:collection1 n:127.0.0.1:55461_i_q%2Fp r:core_node1) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1965593 INFO  (coreZkRegister-2624-thread-1-processing-c:control_collection s:shard1 x:collection1 n:127.0.0.1:55461_i_q%2Fp r:core_node1) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:55461/i_q/p/collection1/ has no replicas
   [junit4]   2> 1965594 INFO  (coreZkRegister-2624-thread-1-processing-c:control_collection s:shard1 x:collection1 n:127.0.0.1:55461_i_q%2Fp r:core_node1) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1965597 INFO  (coreZkRegister-2624-thread-1-processing-c:control_collection s:shard1 x:collection1 n:127.0.0.1:55461_i_q%2Fp r:core_node1) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1965599 INFO  (coreZkRegister-2624-thread-1-processing-c:control_collection s:shard1 x:collection1 n:127.0.0.1:55461_i_q%2Fp r:core_node1) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:55461/i_q/p/collection1/ shard1
   [junit4]   2> 1965600 INFO  (OverseerStateUpdate-95203506598772741-127.0.0.1:55461_i_q%2Fp-n_0000000000) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"https://127.0.0.1:55461/i_q/p",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1965600 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1965601 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1965603 INFO  (zkCallback-917-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@399c19e name:ZooKeeperConnection Watcher:127.0.0.1:56217/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1965603 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1965603 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1965603 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1965607 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1965607 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 1965607 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1965608 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1965612 INFO  (zkCallback-918-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7e87360e name:ZooKeeperConnection Watcher:127.0.0.1:56217/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1965613 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1965613 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1965616 INFO  (OverseerStateUpdate-95203506598772741-127.0.0.1:55461_i_q%2Fp-n_0000000000) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 1
   [junit4]   2> 1965616 INFO  (OverseerStateUpdate-95203506598772741-127.0.0.1:55461_i_q%2Fp-n_0000000000) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1965616 INFO  (zkCallback-915-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1965617 INFO  (zkCallback-917-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1965650 INFO  (coreZkRegister-2624-thread-1-processing-c:control_collection s:shard1 x:collection1 n:127.0.0.1:55461_i_q%2Fp r:core_node1) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are https://127.0.0.1:55461/i_q/p/collection1/ and leader is https://127.0.0.1:55461/i_q/p/collection1/
   [junit4]   2> 1965650 INFO  (coreZkRegister-2624-thread-1-processing-c:control_collection s:shard1 x:collection1 n:127.0.0.1:55461_i_q%2Fp r:core_node1) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:55461/i_q/p
   [junit4]   2> 1965650 INFO  (coreZkRegister-2624-thread-1-processing-c:control_collection s:shard1 x:collection1 n:127.0.0.1:55461_i_q%2Fp r:core_node1) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1965651 INFO  (coreZkRegister-2624-thread-1-processing-c:control_collection s:shard1 x:collection1 n:127.0.0.1:55461_i_q%2Fp r:core_node1) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1965651 INFO  (coreZkRegister-2624-thread-1-processing-c:control_collection s:shard1 x:collection1 n:127.0.0.1:55461_i_q%2Fp r:core_node1) [n:127.0.0.1:55461_i_q%2Fp c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1965653 INFO  (OverseerStateUpdate-95203506598772741-127.0.0.1:55461_i_q%2Fp-n_0000000000) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "base_url":"https://127.0.0.1:55461/i_q/p",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "node_name":"127.0.0.1:55461_i_q%2Fp",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"2"} current state version: 2
   [junit4]   2> 1965654 INFO  (OverseerStateUpdate-95203506598772741-127.0.0.1:55461_i_q%2Fp-n_0000000000) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "base_url":"https://127.0.0.1:55461/i_q/p",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "node_name":"127.0.0.1:55461_i_q%2Fp",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"2"}
   [junit4]   2> 1965654 INFO  (OverseerStateUpdate-95203506598772741-127.0.0.1:55461_i_q%2Fp-n_0000000000) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 1965760 INFO  (zkCallback-917-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1965761 INFO  (zkCallback-915-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1966055 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/shard-1-001/cores/collection1
   [junit4]   2> 1966076 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/shard-1-001
   [junit4]   2> 1966077 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1966105 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6c950708{/i_q/p,null,AVAILABLE}
   [junit4]   2> 1966111 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2c00d5e9{SSL-http/1.1}{127.0.0.1:51792}
   [junit4]   2> 1966111 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.e.j.s.Server Started @1968764ms
   [junit4]   2> 1966111 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/tempDir-001/jetty1, hostPort=51792, hostContext=/i_q/p, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/shard-1-001/cores}
   [junit4]   2> 1966112 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1966112 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/shard-1-001'
   [junit4]   2> 1966112 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1966112 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1966112 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1966113 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1966115 INFO  (zkCallback-919-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@14c3c278 name:ZooKeeperConnection Watcher:127.0.0.1:56217/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1966115 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1966116 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1966117 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1966117 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/shard-1-001/solr.xml
   [junit4]   2> 1966132 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/shard-1-001/cores
   [junit4]   2> 1966132 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.CoreContainer New CoreContainer 1937557899
   [junit4]   2> 1966132 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/shard-1-001]
   [junit4]   2> 1966132 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/shard-1-001/lib
   [junit4]   2> 1966133 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1966135 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1966136 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1966136 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1966136 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56217/solr
   [junit4]   2> 1966136 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1966136 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1966137 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1966138 INFO  (zkCallback-921-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@253cf1cd name:ZooKeeperConnection Watcher:127.0.0.1:56217 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1966139 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1966140 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1966142 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:51792_i_q%2Fp    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1966144 INFO  (zkCallback-922-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3a6d68ec name:ZooKeeperConnection Watcher:127.0.0.1:56217/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1966145 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:51792_i_q%2Fp    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1966155 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:51792_i_q%2Fp    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1967175 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:51792_i_q%2Fp    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51792_i_q%2Fp
   [junit4]   2> 1967175 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:51792_i_q%2Fp    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:51792_i_q%2Fp
   [junit4]   2> 1967176 INFO  (zkCallback-922-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1967176 INFO  (zkCallback-917-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1967176 INFO  (zkCallback-915-thread-1-processing-n:127.0.0.1:55461_i_q%2Fp) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1967178 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:51792_i_q%2Fp    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1967179 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:51792_i_q%2Fp    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95203506598772746-127.0.0.1:51792_i_q%2Fp-n_0000000001
   [junit4]   2> 1967179 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:51792_i_q%2Fp    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/95203506598772741-127.0.0.1:55461_i_q%2Fp-n_0000000000 to know if I could be the leader
   [junit4]   2> 1967204 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:51792_i_q%2Fp    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1967204 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:51792_i_q%2Fp    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1967205 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:51792_i_q%2Fp    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/shard-1-001/cores
   [junit4]   2> 1967206 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:51792_i_q%2Fp    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, configSetProperties=configsetprops.json, collection=collection1, transient=false, config=solrconfig.xml, dataDir=data/, loadOnStartup=true, schema=schema.xml, coreNodeName=, shard=}
   [junit4]   2> 1967206 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:51792_i_q%2Fp    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/shard-1-001/cores/collection1
   [junit4]   2> 1967206 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:51792_i_q%2Fp    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1967207 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:51792_i_q%2Fp    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1967207 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[95EB1EF282912922]) [n:127.0.0.1:51792_i_q%2Fp    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1967208 INFO  (coreLoadExecutor-2641-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1967208 INFO  (coreLoadExecutor-2641-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1967210 INFO  (coreLoadExecutor-2641-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 1967210 INFO  (coreLoadExecutor-2641-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch [collection1]
   [junit4]   2> 1967211 INFO  (coreLoadExecutor-2641-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for [collection1] ver [0]
   [junit4]   2> 1967211 INFO  (OverseerStateUpdate-95203506598772741-127.0.0.1:55461_i_q%2Fp-n_0000000000) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:51792/i_q/p",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:51792_i_q%2Fp",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"2"} current state version: 3
   [junit4]   2> 1967211 INFO  (coreLoadExecutor-2641-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1967212 INFO  (OverseerStateUpdate-95203506598772741-127.0.0.1:55461_i_q%2Fp-n_0000000000) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:51792/i_q/p",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:51792_i_q%2Fp",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"2"}
   [junit4]   2> 1967212 INFO  (OverseerStateUpdate-95203506598772741-127.0.0.1:55461_i_q%2Fp-n_0000000000) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=2
   [junit4]   2> 1967212 INFO  (OverseerStateUpdate-95203506598772741-127.0.0.1:55461_i_q%2Fp-n_0000000000) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 1967315 INFO  (OverseerStateUpdate-95203506598772741-127.0.0.1:55461_i_q%2Fp-n_0000000000) [n:127.0.0.1:55461_i_q%2Fp    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 1967318 INFO  (zkCallback-922-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1967321 INFO  (zkCallback-922-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp    ] o.a.s.c.c.ZkStateReader Updating data for [collection1] from [0] to [1]
   [junit4]   2> 1968211 INFO  (coreLoadExecutor-2641-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1968211 INFO  (coreLoadExecutor-2641-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1968212 INFO  (coreLoadExecutor-2641-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1968212 INFO  (coreLoadExecutor-2641-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/collection1]
   [junit4]   2> 1968213 INFO  (coreLoadExecutor-2641-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/collection1] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1968213 INFO  (coreLoadExecutor-2641-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/shard-1-001/cores/collection1'
   [junit4]   2> 1968213 INFO  (coreLoadExecutor-2641-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1968213 INFO  (coreLoadExecutor-2641-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1968215 INFO  (coreLoadExecutor-2641-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1968224 INFO  (coreLoadExecutor-2641-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1968248 INFO  (coreLoadExecutor-2641-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1968295 INFO  (coreLoadExecutor-2641-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp c:collection1   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1968303 INFO  (coreLoadExecutor-2641-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1968649 INFO  (coreLoadExecutor-2641-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1968659 INFO  (coreLoadExecutor-2641-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1968661 INFO  (coreLoadExecutor-2641-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1968673 INFO  (coreLoadExecutor-2641-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1968677 INFO  (coreLoadExecutor-2641-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1968679 INFO  (coreLoadExecutor-2641-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1968680 INFO  (coreLoadExecutor-2641-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1968682 INFO  (coreLoadExecutor-2641-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp c:collection1   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1968682 INFO  (coreLoadExecutor-2641-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 1968682 INFO  (coreLoadExecutor-2641-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1968682 INFO  (coreLoadExecutor-2641-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/shard-1-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_95EB1EF282912922-001/shard-1-001/cores/collection1/data/]
   [junit4]   2> 1968682 INFO  (coreLoadExecutor-2641-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@385e0377
   [junit4]   2> 1968683 INFO  (coreLoadExecutor-2641-thread-1-processing-n:127.0.0.1:51792_i_q%2Fp) [n:127.0.0.1:51792_i_q%2Fp c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for 

[...truncated too long message...]

.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=it -Dtests.timezone=CST -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR    640s J1 | CollectionsAPIDistributedZkTest.test <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=33805, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:51792/i_q/p: collection already exists: awholynewstresscollection_collection1_5
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([95EB1EF282912922]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:881)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=33808, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:51792/i_q/p: Could not find collection : awholynewstresscollection_collection4_4
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([95EB1EF282912922]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)Throwable #3: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=33809, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:51792/i_q/p: collection already exists: awholynewstresscollection_collection5_8
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([95EB1EF282912922]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:881)
   [junit4]   2> 2603449 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[95EB1EF282912922]-worker) [n:127.0.0.1:38304_i_q%2Fp c:addReplicaColl s:shard2 r:core_node6 x:addReplicaColl_shard2_replica3] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: test params are: codec=FastCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST, chunkSize=5, maxDocsPerChunk=7, blockSize=551), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST, chunkSize=5, blockSize=551)), sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=it, timezone=CST
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=67383416,total=468189184
   [junit4]   2> NOTE: All tests run in this JVM: [ConvertedLegacyTest, TestReqParamsAPI, CSVRequestHandlerTest, TestSolrDeletionPolicy2, TestHdfsUpdateLog, TestSurroundQueryParser, ChangedSchemaMergeTest, DistributedMLTComponentTest, HdfsChaosMonkeySafeLeaderTest, HttpPartitionTest, TestDocumentBuilder, TestClassNameShortening, SolrCmdDistributorTest, HdfsSyncSliceTest, DataDrivenBlockJoinTest, TestConfigSets, WordBreakSolrSpellCheckerTest, AnalyticsQueryTest, EnumFieldTest, TestNoOpRegenerator, TestRequestForwarding, AssignTest, TestLeaderElectionZkExpiry, DirectSolrSpellCheckerTest, HLLUtilTest, TestConfigOverlay, ZkSolrClientTest, MBeansHandlerTest, TestObjectReleaseTracker, DistribCursorPagingTest, FastVectorHighlighterTest, TestHashPartitioner, TestLRUStatsCache, TestSha256AuthenticationProvider, NotRequiredUniqueKeyTest, TestDistribDocBasedVersion, TestStressVersions, CheckHdfsIndexTest, BlockDirectoryTest, TestTrie, ShardRoutingTest, TestRandomRequestDistribution, TestBinaryResponseWriter, TimeZoneUtilsTest, ClusterStateUpdateTest, RecoveryAfterSoftCommitTest, DeleteLastCustomShardedReplicaTest, ExitableDirectoryReaderTest, SpellingQueryConverterTest, TestInitQParser, TestBM25SimilarityFactory, TestMiniSolrCloudClusterBase, TestLMJelinekMercerSimilarityFactory, TestQueryUtils, TestSmileRequest, TestDistributedStatsComponentCardinality, TestDistribIDF, TestReloadAndDeleteDocs, TestDFRSimilarityFactory, TestFastLRUCache, SpellCheckCollatorTest, IndexBasedSpellCheckerTest, TestUpdate, TestSolrQueryParserDefaultOperatorResource, TestStressUserVersions, SharedFSAutoReplicaFailoverUtilsTest, TestUseDocValuesAsStored, CollectionsAPIAsyncDistributedZkTest, OpenExchangeRatesOrgProviderTest, UpdateRequestProcessorFactoryTest, OverseerRolesTest, TestDynamicFieldCollectionResource, ShardRoutingCustomTest, NumericFieldsTest, TestPartialUpdateDeduplication, ChaosMonkeyNothingIsSafeTest, DistributedFacetPivotLargeTest, ZkStateReaderTest, TestJettySolrRunner, TestTrackingShardHandlerFactory, TestLFUCache, RestartWhileUpdatingTest, TestSortByMinMaxFunction, TestQueryTypes, TestSchemaSimilarityResource, TermsComponentTest, TestCloudPivotFacet, TestIntervalFaceting, OutOfBoxZkACLAndCredentialsProvidersTest, CollectionsAPIDistributedZkTest]
   [junit4] Completed [283/564 (1!)] on J1 in 641.11s, 1 test, 1 error <<< FAILURES!

[...truncated 916 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:532: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1444: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1000: There were test failures: 564 suites (7 ignored), 2253 tests, 1 error, 33 ignored (18 assumptions) [seed: 95EB1EF282912922]

Total time: 202 minutes 59 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1073 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1073/

4 tests failed.
FAILED:  org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=10854, name=Thread-9276, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=10854, name=Thread-9276, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
	at __randomizedtesting.SeedInfo.seed([FD9052E9725AF675:75C46D33DCA69B8D]:0)
Caused by: java.lang.AssertionError: 1
	at __randomizedtesting.SeedInfo.seed([FD9052E9725AF675]:0)
	at org.apache.solr.core.CachingDirectoryFactory.close(CachingDirectoryFactory.java:202)
	at org.apache.solr.core.SolrCore.close(SolrCore.java:1255)
	at org.apache.solr.core.SolrCores.close(SolrCores.java:125)
	at org.apache.solr.core.CoreContainer.shutdown(CoreContainer.java:582)
	at org.apache.solr.servlet.SolrDispatchFilter.destroy(SolrDispatchFilter.java:175)
	at org.eclipse.jetty.servlet.FilterHolder.destroyInstance(FilterHolder.java:173)
	at org.eclipse.jetty.servlet.FilterHolder.doStop(FilterHolder.java:151)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
	at org.eclipse.jetty.servlet.ServletHandler.doStop(ServletHandler.java:241)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:143)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:162)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStop(AbstractHandler.java:73)
	at org.eclipse.jetty.server.session.SessionHandler.doStop(SessionHandler.java:127)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:143)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:162)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStop(AbstractHandler.java:73)
	at org.eclipse.jetty.server.handler.ContextHandler.doStop(ContextHandler.java:835)
	at org.eclipse.jetty.servlet.ServletContextHandler.doStop(ServletContextHandler.java:215)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:143)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:162)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStop(AbstractHandler.java:73)
	at org.eclipse.jetty.server.Server.doStop(Server.java:456)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.stop(JettySolrRunner.java:444)
	at org.apache.solr.cloud.ChaosMonkey.kill(ChaosMonkey.java:232)
	at org.apache.solr.cloud.ChaosMonkey.kill(ChaosMonkey.java:242)
	at org.apache.solr.cloud.ChaosMonkey.killJetty(ChaosMonkey.java:175)
	at org.apache.solr.cloud.ChaosMonkey.killRandomShard(ChaosMonkey.java:336)
	at org.apache.solr.cloud.ChaosMonkey.killRandomShard(ChaosMonkey.java:321)
	at org.apache.solr.cloud.ChaosMonkey$2.run(ChaosMonkey.java:519)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderTest

Error Message:
ObjectTracker found 2 object(s) that were not released!!! [SolrCore, MMapDirectory]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 2 object(s) that were not released!!! [SolrCore, MMapDirectory]
	at __randomizedtesting.SeedInfo.seed([FD9052E9725AF675]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:228)
	at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest:     1) Thread[id=10782, name=org.eclipse.jetty.server.session.HashSessionManager@38de084bTimer, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest: 
   1) Thread[id=10782, name=org.eclipse.jetty.server.session.HashSessionManager@38de084bTimer, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([FD9052E9725AF675]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=10782, name=org.eclipse.jetty.server.session.HashSessionManager@38de084bTimer, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=10782, name=org.eclipse.jetty.server.session.HashSessionManager@38de084bTimer, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([FD9052E9725AF675]:0)




Build Log:
[...truncated 10338 lines...]
   [junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/init-core-data-001
   [junit4]   2> 463255 INFO  (SUITE-ChaosMonkeySafeLeaderTest-seed#[FD9052E9725AF675]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 463256 INFO  (SUITE-ChaosMonkeySafeLeaderTest-seed#[FD9052E9725AF675]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /t_dj/
   [junit4]   2> 463261 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 463262 INFO  (Thread-9181) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 463262 INFO  (Thread-9181) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 463362 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.ZkTestServer start zk server on port:43709
   [junit4]   2> 463362 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 463363 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 463365 INFO  (zkCallback-227-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5bc935f6 name:ZooKeeperConnection Watcher:127.0.0.1:43709 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 463365 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 463365 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 463365 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 463367 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 463368 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 463370 INFO  (zkCallback-228-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4decee20 name:ZooKeeperConnection Watcher:127.0.0.1:43709/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 463370 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 463371 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 463371 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 463374 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 463376 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 463378 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 463380 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 463380 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 463383 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 463383 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 463385 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 463385 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 463387 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 463387 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 463389 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 463389 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 463392 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 463392 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 463393 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 463393 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 463395 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 463396 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 463402 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 463402 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 463403 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 463404 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 463405 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 463406 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 463739 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/control-001/cores/collection1
   [junit4]   2> 463741 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 463743 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@df04ef7{/t_dj,null,AVAILABLE}
   [junit4]   2> 463749 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.e.j.s.ServerConnector Started ServerConnector@1d2ebe5b{HTTP/1.1}{127.0.0.1:49318}
   [junit4]   2> 463749 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.e.j.s.Server Started @466840ms
   [junit4]   2> 463749 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/t_dj, hostPort=49318, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/control-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/tempDir-001/control/data}
   [junit4]   2> 463750 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4dfd245f
   [junit4]   2> 463750 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/control-001'
   [junit4]   2> 463750 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 463750 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 463751 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 463751 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 463753 INFO  (zkCallback-229-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@23d0c01a name:ZooKeeperConnection Watcher:127.0.0.1:43709/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 463753 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 463753 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 463755 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 463755 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/control-001/solr.xml
   [junit4]   2> 463775 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/control-001/cores
   [junit4]   2> 463775 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.CoreContainer New CoreContainer 612910795
   [junit4]   2> 463775 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/control-001]
   [junit4]   2> 463775 WARN  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/control-001/lib
   [junit4]   2> 463776 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 463777 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 463778 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 463778 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 463778 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43709/solr
   [junit4]   2> 463778 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 463778 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 463778 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 463780 INFO  (zkCallback-231-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4dea2663 name:ZooKeeperConnection Watcher:127.0.0.1:43709 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 463780 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 463781 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 463783 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 463784 INFO  (zkCallback-232-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@58022901 name:ZooKeeperConnection Watcher:127.0.0.1:43709/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 463784 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 463785 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 463787 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 463791 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 463795 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 463797 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 463802 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 463805 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 463806 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 463807 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 463809 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49318_t_dj
   [junit4]   2> 463809 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:49318_t_dj
   [junit4]   2> 463811 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 463812 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 463815 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 463816 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95197624551079940-127.0.0.1:49318_t_dj-n_0000000000
   [junit4]   2> 463817 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:49318_t_dj
   [junit4]   2> 463817 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 463819 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.Overseer Overseer (id=95197624551079940-127.0.0.1:49318_t_dj-n_0000000000) starting
   [junit4]   2> 463822 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 463829 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 463831 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 463831 INFO  (OverseerCollectionConfigSetProcessor-95197624551079940-127.0.0.1:49318_t_dj-n_0000000000) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 463832 INFO  (OverseerStateUpdate-95197624551079940-127.0.0.1:49318_t_dj-n_0000000000) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 463847 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 463847 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 463849 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/control-001/cores
   [junit4]   2> 463850 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {coreNodeName=, dataDir=data/, loadOnStartup=true, shard=, collection=control_collection, configSetProperties=configsetprops.json, name=collection1, transient=false, schema=schema.xml, config=solrconfig.xml}
   [junit4]   2> 463850 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/control-001/cores/collection1
   [junit4]   2> 463850 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 463851 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:49318_t_dj    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 463852 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:49318_t_dj    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 463853 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 463853 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 463856 INFO  (OverseerStateUpdate-95197624551079940-127.0.0.1:49318_t_dj-n_0000000000) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:49318/t_dj",
   [junit4]   2>   "node_name":"127.0.0.1:49318_t_dj",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1"} current state version: 0
   [junit4]   2> 463856 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 463857 INFO  (OverseerStateUpdate-95197624551079940-127.0.0.1:49318_t_dj-n_0000000000) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:49318/t_dj",
   [junit4]   2>   "node_name":"127.0.0.1:49318_t_dj",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1"}
   [junit4]   2> 463857 INFO  (OverseerStateUpdate-95197624551079940-127.0.0.1:49318_t_dj-n_0000000000) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 463858 INFO  (OverseerStateUpdate-95197624551079940-127.0.0.1:49318_t_dj-n_0000000000) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 463861 INFO  (zkCallback-232-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 464857 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 464857 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 464857 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 464857 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 464858 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 464858 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/control-001/cores/collection1'
   [junit4]   2> 464858 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 464858 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 464860 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 464867 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 464883 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 464916 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 464923 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 465165 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 465166 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 465168 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 465171 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 465197 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 465197 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 465198 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore solr.StandardDirectoryFactory
   [junit4]   2> 465198 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/control-001/cores/collection1/data/]
   [junit4]   2> 465198 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3e4d106f
   [junit4]   2> 465200 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/control-001/cores/collection1/data
   [junit4]   2> 465200 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/control-001/cores/collection1/data/index/
   [junit4]   2> 465200 WARN  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 465201 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/control-001/cores/collection1/data/index
   [junit4]   2> 465201 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=11, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.18342186312589917]
   [junit4]   2> 465209 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/control-001/cores/collection1/data/index,segFN=segments_1,generation=1}
   [junit4]   2> 465209 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 465210 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/control-001/cores/collection1/data/
   [junit4]   2> 465217 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 465217 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 465217 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 465217 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 465217 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 465217 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 465218 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 465218 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 465218 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 465218 WARN  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 465221 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 465223 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 465224 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 465226 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 465231 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /update/json,/dump,standard,/update/csv,/admin/mbeans,/admin/ping,/admin/properties,/admin/segments,/config,/admin/system,/admin/luke,/update/json/docs,/admin/file,/get,/schema,/replication,/admin/threads,/admin/logging,/admin/plugins,/update
   [junit4]   2> 465232 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 465232 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 465232 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 465233 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms; 
   [junit4]   2> 465233 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 465234 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=41, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3436575813588232]
   [junit4]   2> 465234 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/control-001/cores/collection1/data/index,segFN=segments_1,generation=1}
   [junit4]   2> 465234 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 465235 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@3038a16[collection1] main
   [junit4]   2> 465235 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 465235 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 465236 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 465236 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 465236 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 465236 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 465236 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 465236 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 465237 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 465237 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 465237 INFO  (searcherExecutor-685-thread-1-processing-c:control_collection s:shard1 r:core_node1 n:127.0.0.1:49318_t_dj x:collection1) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3038a16[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 465237 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 465238 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 465238 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 465238 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1523161994888216576
   [junit4]   2> 465240 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1523161994888216576
   [junit4]   2> 465240 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 465240 INFO  (coreLoadExecutor-684-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 465241 INFO  (coreZkRegister-678-thread-1-processing-c:control_collection s:shard1 r:core_node1 n:127.0.0.1:49318_t_dj x:collection1) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:49318/t_dj collection:control_collection shard:shard1
   [junit4]   2> 465242 INFO  (coreZkRegister-678-thread-1-processing-c:control_collection s:shard1 r:core_node1 n:127.0.0.1:49318_t_dj x:collection1) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 465245 INFO  (coreZkRegister-678-thread-1-processing-c:control_collection s:shard1 r:core_node1 n:127.0.0.1:49318_t_dj x:collection1) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95197624551079940-core_node1-n_0000000000
   [junit4]   2> 465245 INFO  (coreZkRegister-678-thread-1-processing-c:control_collection s:shard1 r:core_node1 n:127.0.0.1:49318_t_dj x:collection1) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 465247 INFO  (coreZkRegister-678-thread-1-processing-c:control_collection s:shard1 r:core_node1 n:127.0.0.1:49318_t_dj x:collection1) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 465247 INFO  (coreZkRegister-678-thread-1-processing-c:control_collection s:shard1 r:core_node1 n:127.0.0.1:49318_t_dj x:collection1) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 465247 INFO  (coreZkRegister-678-thread-1-processing-c:control_collection s:shard1 r:core_node1 n:127.0.0.1:49318_t_dj x:collection1) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:49318/t_dj/collection1/
   [junit4]   2> 465247 INFO  (coreZkRegister-678-thread-1-processing-c:control_collection s:shard1 r:core_node1 n:127.0.0.1:49318_t_dj x:collection1) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 465247 INFO  (coreZkRegister-678-thread-1-processing-c:control_collection s:shard1 r:core_node1 n:127.0.0.1:49318_t_dj x:collection1) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:49318/t_dj/collection1/ has no replicas
   [junit4]   2> 465247 INFO  (OverseerStateUpdate-95197624551079940-127.0.0.1:49318_t_dj-n_0000000000) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 465248 INFO  (coreZkRegister-678-thread-1-processing-c:control_collection s:shard1 r:core_node1 n:127.0.0.1:49318_t_dj x:collection1) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 465253 INFO  (coreZkRegister-678-thread-1-processing-c:control_collection s:shard1 r:core_node1 n:127.0.0.1:49318_t_dj x:collection1) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 465255 INFO  (coreZkRegister-678-thread-1-processing-c:control_collection s:shard1 r:core_node1 n:127.0.0.1:49318_t_dj x:collection1) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:49318/t_dj/collection1/ shard1
   [junit4]   2> 465255 INFO  (OverseerStateUpdate-95197624551079940-127.0.0.1:49318_t_dj-n_0000000000) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:49318/t_dj",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 465353 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 465354 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 465356 INFO  (zkCallback-234-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@29e982be name:ZooKeeperConnection Watcher:127.0.0.1:43709/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 465356 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 465356 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 465356 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 465358 INFO  (zkCallback-232-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 465358 INFO  (zkCallback-234-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 465359 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 465407 INFO  (coreZkRegister-678-thread-1-processing-c:control_collection s:shard1 r:core_node1 n:127.0.0.1:49318_t_dj x:collection1) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:49318/t_dj/collection1/ and leader is http://127.0.0.1:49318/t_dj/collection1/
   [junit4]   2> 465407 INFO  (coreZkRegister-678-thread-1-processing-c:control_collection s:shard1 r:core_node1 n:127.0.0.1:49318_t_dj x:collection1) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49318/t_dj
   [junit4]   2> 465407 INFO  (coreZkRegister-678-thread-1-processing-c:control_collection s:shard1 r:core_node1 n:127.0.0.1:49318_t_dj x:collection1) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 465407 INFO  (coreZkRegister-678-thread-1-processing-c:control_collection s:shard1 r:core_node1 n:127.0.0.1:49318_t_dj x:collection1) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 465407 INFO  (coreZkRegister-678-thread-1-processing-c:control_collection s:shard1 r:core_node1 n:127.0.0.1:49318_t_dj x:collection1) [n:127.0.0.1:49318_t_dj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 465410 INFO  (OverseerStateUpdate-95197624551079940-127.0.0.1:49318_t_dj-n_0000000000) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:49318/t_dj",
   [junit4]   2>   "node_name":"127.0.0.1:49318_t_dj",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"3"} current state version: 2
   [junit4]   2> 465411 INFO  (OverseerStateUpdate-95197624551079940-127.0.0.1:49318_t_dj-n_0000000000) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:49318/t_dj",
   [junit4]   2>   "node_name":"127.0.0.1:49318_t_dj",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"3"}
   [junit4]   2> 465514 INFO  (zkCallback-234-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 465514 INFO  (zkCallback-232-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 465722 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/shard-1-001/cores/collection1
   [junit4]   2> 465723 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/shard-1-001
   [junit4]   2> 465723 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 465730 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2c89dc26{/t_dj,null,AVAILABLE}
   [junit4]   2> 465730 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.e.j.s.ServerConnector Started ServerConnector@ed459a2{HTTP/1.1}{127.0.0.1:35699}
   [junit4]   2> 465730 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.e.j.s.Server Started @468821ms
   [junit4]   2> 465730 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/shard-1-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostPort=35699, hostContext=/t_dj}
   [junit4]   2> 465731 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4dfd245f
   [junit4]   2> 465731 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/shard-1-001'
   [junit4]   2> 465731 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 465731 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 465731 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 465746 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 465749 INFO  (zkCallback-235-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@70bb9757 name:ZooKeeperConnection Watcher:127.0.0.1:43709/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 465749 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 465749 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 465754 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 465754 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/shard-1-001/solr.xml
   [junit4]   2> 465774 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/shard-1-001/cores
   [junit4]   2> 465774 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.CoreContainer New CoreContainer 1551091913
   [junit4]   2> 465774 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/shard-1-001]
   [junit4]   2> 465775 WARN  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/shard-1-001/lib
   [junit4]   2> 465775 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 465777 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 465777 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 465777 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 465777 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43709/solr
   [junit4]   2> 465777 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 465777 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 465793 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 465797 INFO  (zkCallback-237-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@750697d name:ZooKeeperConnection Watcher:127.0.0.1:43709 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 465797 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 465798 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 465808 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:35699_t_dj    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 465821 INFO  (zkCallback-238-thread-1-processing-n:127.0.0.1:35699_t_dj) [n:127.0.0.1:35699_t_dj    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3437cb0f name:ZooKeeperConnection Watcher:127.0.0.1:43709/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 465821 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:35699_t_dj    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 465828 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:35699_t_dj    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 466836 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:35699_t_dj    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35699_t_dj
   [junit4]   2> 466837 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:35699_t_dj    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:35699_t_dj
   [junit4]   2> 466838 INFO  (zkCallback-232-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 466838 INFO  (zkCallback-234-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 466838 INFO  (zkCallback-238-thread-1-processing-n:127.0.0.1:35699_t_dj) [n:127.0.0.1:35699_t_dj    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 466841 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:35699_t_dj    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 466842 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:35699_t_dj    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95197624551079944-127.0.0.1:35699_t_dj-n_0000000001
   [junit4]   2> 466843 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:35699_t_dj    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/95197624551079940-127.0.0.1:49318_t_dj-n_0000000000 to know if I could be the leader
   [junit4]   2> 466861 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:35699_t_dj    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 466861 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:35699_t_dj    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 466862 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:35699_t_dj    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/shard-1-001/cores
   [junit4]   2> 466863 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:35699_t_dj    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {dataDir=data/, shard=, name=collection1, transient=false, loadOnStartup=true, configSetProperties=configsetprops.json, config=solrconfig.xml, schema=schema.xml, collection=collection1, coreNodeName=}
   [junit4]   2> 466863 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:35699_t_dj    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/shard-1-001/cores/collection1
   [junit4]   2> 466863 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:35699_t_dj    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 466864 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:35699_t_dj    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 466864 INFO  (coreLoadExecutor-695-thread-1-processing-n:127.0.0.1:35699_t_dj) [n:127.0.0.1:35699_t_dj c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 466864 INFO  (TEST-ChaosMonkeySafeLeaderTest.test-seed#[FD9052E9725AF675]) [n:127.0.0.1:35699_t_dj    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 466864 INFO  (coreLoadExecutor-695-thread-1-processing-n:127.0.0.1:35699_t_dj) [n:127.0.0.1:35699_t_dj c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 466866 INFO  (OverseerStateUpdate-95197624551079940-127.0.0.1:49318_t_dj-n_0000000000) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:35699/t_dj",
   [junit4]   2>   "node_name":"127.0.0.1:35699_t_dj",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"3"} current state version: 3
   [junit4]   2> 466866 INFO  (coreLoadExecutor-695-thread-1-processing-n:127.0.0.1:35699_t_dj) [n:127.0.0.1:35699_t_dj c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 466867 INFO  (OverseerStateUpdate-95197624551079940-127.0.0.1:49318_t_dj-n_0000000000) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:35699/t_dj",
   [junit4]   2>   "node_name":"127.0.0.1:35699_t_dj",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"3"}
   [junit4]   2> 466867 INFO  (OverseerStateUpdate-95197624551079940-127.0.0.1:49318_t_dj-n_0000000000) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 466867 INFO  (OverseerStateUpdate-95197624551079940-127.0.0.1:49318_t_dj-n_0000000000) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 466971 INFO  (zkCallback-234-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 466971 INFO  (zkCallback-232-thread-1-processing-n:127.0.0.1:49318_t_dj) [n:127.0.0.1:49318_t_dj    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 466971 INFO  (zkCallback-238-thread-1-processing-n:127.0.0.1:35699_t_dj) [n:127.0.0.1:35699_t_dj    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 467867 INFO  (coreLoadExecutor-695-thread-1-processing-n:127.0.0.1:35699_t_dj) [n:127.0.0.1:35699_t_dj c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 467867 INFO  (coreLoadExecutor-695-thread-1-processing-n:127.0.0.1:35699_t_dj) [n:127.0.0.1:35699_t_dj c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 467868 INFO  (coreLoadExecutor-695-thread-1-processing-n:127.0.0.1:35699_t_dj) [n:127.0.0.1:35699_t_dj c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 467868 INFO  (coreLoadExecutor-695-thread-1-processing-n:127.0.0.1:35699_t_dj) [n:127.0.0.1:35699_t_dj c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/collection1]
   [junit4]   2> 467868 INFO  (coreLoadExecutor-695-thread-1-processing-n:127.0.0.1:35699_t_dj) [n:127.0.0.1:35699_t_dj c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/collection1] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 467868 INFO  (coreLoadExecutor-695-thread-1-processing-n:127.0.0.1:35699_t_dj) [n:127.0.0.1:35699_t_dj c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_FD9052E9725AF675-001/shard-1-001/cores/collection1'
   [junit4]   2> 467869 INFO  (coreLoadExecutor-695-thread-1-processing-n:127.0.0.1:35699_t_dj) [n:127.0.0.1:35699_t_dj c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 467869 INFO  (coreLoadExecutor-695-thread-1-processing-n:127.0.0.1:35699_t_dj) [n:127.0.0.1:35699_t_dj c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 467871 INFO  (coreLoadExecutor-695-thread-1-processing-n:127.0.0.1:35699_t_dj) [n:127.0.0.1:35699_t_dj c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 467877 INFO  (coreLoadExecutor-695-thread-1-processing-n:127.0.0.1:35699_t_dj) [n:127.0.0.1:35699_t_dj c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 467892 INFO  (coreLoadExecutor-695-thread-1-processing-n:127.0.0.1:35699_t_dj) [n:127.0.0.1:35699_t_dj c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 467923 INFO  (coreLoadExecutor-695-thread-1-processing-n:127.0.0.1:35699_t_dj) [n:127.0.0.1:35699_t_dj c:collection1   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 467930 INFO  (coreLoadExecutor-695-thread-1-processing-n:127.0.0.1:35699_t_dj) [n:127.0.0.1:35699_t_dj c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 468146 INFO  (coreLoadExecutor-695-thread-1-processing-n:127.0.0.1:35699_t_dj) [n:127.0.0.1:35699_t_dj c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 468147 INFO  (coreLoadExecutor-695-thread-1-processing-n:127.0.0.1:35699_t_dj) [n:127.0.0.1:35699_t_dj c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 468148 INFO  (coreLoadExecutor-695-thread-1-processing-n:127.0.0.1:35699_t_dj) [n:127.0.0.1:35699_t_dj c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 468152 INFO  (coreLoadExecutor-695-thread-1-processing-n:127.0.0.1:35699_t_dj) [n:127.0.0.1:35699_t_dj c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 468172 INFO  (coreLoadExecutor-695-thread-1-processing-n:127.0.0.1:35699_t_dj) [n:127.0.0.1:35699_t_dj c:collection1   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 468172 INFO  (coreLoadExecutor-695-thread-1-processing-n:127.0.0.1:35699_t_dj) [n:127.0.0.1:35699_t_dj c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 468172 INFO  (coreLoadExecutor-695-thread-1-processing-n:127.0.0.1:35699_t_dj) [n:127.0.0.1:35699_t_dj c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore solr.StandardDirectoryFactory
   [junit4]   2> 468172 INFO  (coreLoadExecutor-695-thread-1-processing-n:127.0.0.1:35699_t_dj) [n:127.0.0.1:35699_t_dj c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [

[...truncated too long message...]

pache.solr.core.CoreContainer$1.call(CoreContainer.java:458)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:232)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> Jan 12, 2016 12:10:53 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> Jan 12, 2016 12:11:13 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest: 
   [junit4]   2>    1) Thread[id=10782, name=org.eclipse.jetty.server.session.HashSessionManager@38de084bTimer, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Jan 12, 2016 12:11:13 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=10782, name=org.eclipse.jetty.server.session.HashSessionManager@38de084bTimer, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
   [junit4]   2> Jan 12, 2016 12:11:16 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> SEVERE: There are still zombie threads that couldn't be terminated:
   [junit4]   2>    1) Thread[id=10782, name=org.eclipse.jetty.server.session.HashSessionManager@38de084bTimer, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {a_i=PostingsFormat(name=LuceneVarGapDocFreqInterval), id=PostingsFormat(name=LuceneVarGapDocFreqInterval), rnd_b=PostingsFormat(name=LuceneVarGapDocFreqInterval), a_t=FSTOrd50, _version_=PostingsFormat(name=Memory doPackFST= true)}, docValues:{}, sim=DefaultSimilarity, locale=ja, timezone=US/Mountain
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=2,free=104598952,total=429391872
   [junit4]   2> NOTE: All tests run in this JVM: [FastVectorHighlighterTest, LoggingHandlerTest, MBeansHandlerTest, TestCustomSort, TestDistributedStatsComponentCardinality, DOMUtilTest, HdfsWriteToMultipleCollectionsTest, MinimalSchemaTest, ChaosMonkeyNothingIsSafeTest, SolrXmlInZkTest, UniqFieldsUpdateProcessorFactoryTest, BasicAuthIntegrationTest, TestReloadAndDeleteDocs, WordBreakSolrSpellCheckerTest, TestExactSharedStatsCache, IndexBasedSpellCheckerTest, TestSolrCloudWithKerberosAlt, SpellCheckComponentTest, TestCustomDocTransformer, TestBlendedInfixSuggestions, TestSurroundQueryParser, TestRequestForwarding, ChaosMonkeySafeLeaderTest]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ChaosMonkeySafeLeaderTest -Dtests.seed=FD9052E9725AF675 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=ja -Dtests.timezone=US/Mountain -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J1 | ChaosMonkeySafeLeaderTest (suite) <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: ObjectTracker found 2 object(s) that were not released!!! [SolrCore, MMapDirectory]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([FD9052E9725AF675]:0)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:228)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest: 
   [junit4]    >    1) Thread[id=10782, name=org.eclipse.jetty.server.session.HashSessionManager@38de084bTimer, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([FD9052E9725AF675]:0)Throwable #3: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=10782, name=org.eclipse.jetty.server.session.HashSessionManager@38de084bTimer, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([FD9052E9725AF675]:0)
   [junit4] Completed [74/563 (1!)] on J1 in 203.89s, 1 test, 1 failure, 3 errors <<< FAILURES!

[...truncated 1504 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1444: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1000: There were test failures: 563 suites (7 ignored), 2236 tests, 3 suite-level errors, 1 error, 1403 ignored (7 assumptions) [seed: FD9052E9725AF675]

Total time: 166 minutes 32 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1072 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1072/

2 tests failed.
FAILED:  org.apache.solr.core.HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig

Error Message:
The max direct memory is likely too low.  Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~134.217728 MB.

Stack Trace:
java.lang.RuntimeException: The max direct memory is likely too low.  Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~134.217728 MB.
	at __randomizedtesting.SeedInfo.seed([8F089C32D5F6338D:78A75519287FD9A6]:0)
	at org.apache.solr.core.HdfsDirectoryFactory.createBlockCache(HdfsDirectoryFactory.java:296)
	at org.apache.solr.core.HdfsDirectoryFactory.getBlockDirectoryCache(HdfsDirectoryFactory.java:272)
	at org.apache.solr.core.HdfsDirectoryFactory.create(HdfsDirectoryFactory.java:213)
	at org.apache.solr.core.HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig(HdfsDirectoryFactoryTest.java:114)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.OutOfMemoryError: Direct buffer memory
	at java.nio.Bits.reserveMemory(Bits.java:658)
	at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
	at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306)
	at org.apache.solr.store.blockcache.BlockCache.<init>(BlockCache.java:64)
	at org.apache.solr.core.HdfsDirectoryFactory.createBlockCache(HdfsDirectoryFactory.java:294)
	... 42 more


FAILED:  org.apache.solr.index.hdfs.CheckHdfsIndexTest.doTest

Error Message:
Could not find a healthy node to handle the request.

Stack Trace:
org.apache.solr.common.SolrException: Could not find a healthy node to handle the request.
	at __randomizedtesting.SeedInfo.seed([8F089C32D5F6338D:284C2496B84D2034]:0)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1085)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:954)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:954)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:954)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:954)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:954)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
	at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:483)
	at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:464)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.commit(AbstractFullDistribZkTestBase.java:1505)
	at org.apache.solr.index.hdfs.CheckHdfsIndexTest.doTest(CheckHdfsIndexTest.java:101)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:965)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:940)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10830 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20160111_120123_535.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid21414.hprof ...
   [junit4] Heap dump file created [609670035 bytes in 5.932 secs]
   [junit4] <<< JVM J1: EOF ----

   [junit4] JVM J1: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20160111_120123_535.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] WARN: Unhandled exception in event serialization. -> java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] <<< JVM J1: EOF ----

[...truncated 602 lines...]
   [junit4] Suite: org.apache.solr.core.HdfsDirectoryFactoryTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.HdfsDirectoryFactoryTest_8F089C32D5F6338D-001/init-core-data-001
   [junit4]   2> 4173609 INFO  (SUITE-HdfsDirectoryFactoryTest-seed#[8F089C32D5F6338D]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true)
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 4173644 WARN  (SUITE-HdfsDirectoryFactoryTest-seed#[8F089C32D5F6338D]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 4173650 WARN  (SUITE-HdfsDirectoryFactoryTest-seed#[8F089C32D5F6338D]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 4173652 INFO  (SUITE-HdfsDirectoryFactoryTest-seed#[8F089C32D5F6338D]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 4173666 INFO  (SUITE-HdfsDirectoryFactoryTest-seed#[8F089C32D5F6338D]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_53663_hdfs____.sab96p/webapp
   [junit4]   2> 4173781 INFO  (SUITE-HdfsDirectoryFactoryTest-seed#[8F089C32D5F6338D]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 4173943 INFO  (SUITE-HdfsDirectoryFactoryTest-seed#[8F089C32D5F6338D]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:53663
   [junit4]   2> 4173998 WARN  (SUITE-HdfsDirectoryFactoryTest-seed#[8F089C32D5F6338D]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 4173999 INFO  (SUITE-HdfsDirectoryFactoryTest-seed#[8F089C32D5F6338D]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 4174016 INFO  (SUITE-HdfsDirectoryFactoryTest-seed#[8F089C32D5F6338D]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_36910_datanode____e2cc3g/webapp
   [junit4]   2> 4174133 INFO  (SUITE-HdfsDirectoryFactoryTest-seed#[8F089C32D5F6338D]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 4174341 INFO  (SUITE-HdfsDirectoryFactoryTest-seed#[8F089C32D5F6338D]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36910
   [junit4]   2> 4174371 WARN  (SUITE-HdfsDirectoryFactoryTest-seed#[8F089C32D5F6338D]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 4174373 INFO  (SUITE-HdfsDirectoryFactoryTest-seed#[8F089C32D5F6338D]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 4174386 INFO  (SUITE-HdfsDirectoryFactoryTest-seed#[8F089C32D5F6338D]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_41924_datanode____.k4zgcb/webapp
   [junit4]   2> 4174411 INFO  (IPC Server handler 3 on 58558) [    ] BlockStateChange BLOCK* processReport: from storage DS-6acc99d6-5839-4162-bd2d-c8063b375e41 node DatanodeRegistration(127.0.0.1, datanodeUuid=10416d2b-5638-40bc-a743-3ecf129c4740, infoPort=36910, ipcPort=39544, storageInfo=lv=-56;cid=testClusterID;nsid=1325544583;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 4174411 INFO  (IPC Server handler 3 on 58558) [    ] BlockStateChange BLOCK* processReport: from storage DS-9bcd3ee7-a123-4e9a-bf10-ffe94cbbdf2d node DatanodeRegistration(127.0.0.1, datanodeUuid=10416d2b-5638-40bc-a743-3ecf129c4740, infoPort=36910, ipcPort=39544, storageInfo=lv=-56;cid=testClusterID;nsid=1325544583;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 4174509 INFO  (SUITE-HdfsDirectoryFactoryTest-seed#[8F089C32D5F6338D]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 4174715 INFO  (SUITE-HdfsDirectoryFactoryTest-seed#[8F089C32D5F6338D]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41924
   [junit4]   2> 4174787 INFO  (IPC Server handler 8 on 58558) [    ] BlockStateChange BLOCK* processReport: from storage DS-9ab7abe7-74aa-41c0-a006-f29ef21d5d03 node DatanodeRegistration(127.0.0.1, datanodeUuid=12802cc4-6db1-4ef5-8642-74a794f9f7e4, infoPort=41924, ipcPort=46508, storageInfo=lv=-56;cid=testClusterID;nsid=1325544583;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 4174787 INFO  (IPC Server handler 8 on 58558) [    ] BlockStateChange BLOCK* processReport: from storage DS-0ae8619b-5584-4676-91c1-ab5b358cf654 node DatanodeRegistration(127.0.0.1, datanodeUuid=12802cc4-6db1-4ef5-8642-74a794f9f7e4, infoPort=41924, ipcPort=46508, storageInfo=lv=-56;cid=testClusterID;nsid=1325544583;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 4174881 INFO  (TEST-HdfsDirectoryFactoryTest.testCleanupOldIndexDirectories-seed#[8F089C32D5F6338D]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testCleanupOldIndexDirectories
   [junit4]   2> 4174881 INFO  (TEST-HdfsDirectoryFactoryTest.testCleanupOldIndexDirectories-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:58558/solr1
   [junit4]   2> 4174881 INFO  (TEST-HdfsDirectoryFactoryTest.testCleanupOldIndexDirectories-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 4174881 INFO  (TEST-HdfsDirectoryFactoryTest.testCleanupOldIndexDirectories-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.CoreContainer New CoreContainer 705101462
   [junit4]   2> 4174881 INFO  (TEST-HdfsDirectoryFactoryTest.testCleanupOldIndexDirectories-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {configSetProperties=configsetprops.json, transient=false, schema=schema.xml, name=mock, config=solrconfig.xml, loadOnStartup=true, dataDir=data/}
   [junit4]   2> 4174900 INFO  (TEST-HdfsDirectoryFactoryTest.testCleanupOldIndexDirectories-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.HdfsDirectoryFactory Deleted old index directory hdfs://localhost:58558/solr1/mock/data/index.20160111141101588
   [junit4]   2> 4174901 INFO  (TEST-HdfsDirectoryFactoryTest.testCleanupOldIndexDirectories-seed#[8F089C32D5F6338D]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testCleanupOldIndexDirectories
   [junit4]   2> 4174938 INFO  (TEST-HdfsDirectoryFactoryTest.testLocalityReporter-seed#[8F089C32D5F6338D]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testLocalityReporter
   [junit4]   2> 4174947 INFO  (TEST-HdfsDirectoryFactoryTest.testLocalityReporter-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:58558/solr
   [junit4]   2> 4174948 INFO  (TEST-HdfsDirectoryFactoryTest.testLocalityReporter-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 4174948 INFO  (TEST-HdfsDirectoryFactoryTest.testLocalityReporter-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:58558/solr3/
   [junit4]   2> 4174960 INFO  (TEST-HdfsDirectoryFactoryTest.testLocalityReporter-seed#[8F089C32D5F6338D]) [    ] o.a.s.s.h.HdfsLocalityReporter Registering direcotry hdfs://localhost:58558/solr3 for locality metrics.
   [junit4]   2> 4174971 INFO  (IPC Server handler 1 on 58558) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:38183 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9bcd3ee7-a123-4e9a-bf10-ffe94cbbdf2d:NORMAL:127.0.0.1:52681|RBW], ReplicaUnderConstruction[[DISK]DS-9ab7abe7-74aa-41c0-a006-f29ef21d5d03:NORMAL:127.0.0.1:38183|RBW]]} size 0
   [junit4]   2> 4174971 INFO  (IPC Server handler 2 on 58558) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:52681 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9bcd3ee7-a123-4e9a-bf10-ffe94cbbdf2d:NORMAL:127.0.0.1:52681|RBW], ReplicaUnderConstruction[[DISK]DS-9ab7abe7-74aa-41c0-a006-f29ef21d5d03:NORMAL:127.0.0.1:38183|RBW]]} size 0
   [junit4]   2> 4174976 INFO  (TEST-HdfsDirectoryFactoryTest.testLocalityReporter-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.CachingDirectoryFactory Closing HdfsDirectoryFactory - 0 directories currently being tracked
   [junit4]   2> 4174976 INFO  (TEST-HdfsDirectoryFactoryTest.testLocalityReporter-seed#[8F089C32D5F6338D]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testLocalityReporter
   [junit4]   2> 4175014 INFO  (TEST-HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig-seed#[8F089C32D5F6338D]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testInitArgsOrSysPropConfig
   [junit4]   2> 4175014 INFO  (TEST-HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:58558/solr1
   [junit4]   2> 4175014 INFO  (TEST-HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 4175015 INFO  (TEST-HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.CoreContainer New CoreContainer 1804671906
   [junit4]   2> 4175015 INFO  (TEST-HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {configSetProperties=configsetprops.json, transient=false, schema=schema.xml, name=mock, loadOnStartup=true, config=solrconfig.xml, dataDir=data/}
   [junit4]   2> 4175015 INFO  (TEST-HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:58558/solr2
   [junit4]   2> 4175015 INFO  (TEST-HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 4175015 INFO  (TEST-HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.CoreContainer New CoreContainer 61693914
   [junit4]   2> 4175015 INFO  (TEST-HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {config=solrconfig.xml, configSetProperties=configsetprops.json, schema=schema.xml, transient=false, dataDir=data/, loadOnStartup=true, name=mock}
   [junit4]   2> 4175016 INFO  (TEST-HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:58558/solr2
   [junit4]   2> 4175016 INFO  (TEST-HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 4175016 INFO  (TEST-HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.CoreContainer New CoreContainer 1802494744
   [junit4]   2> 4175016 INFO  (TEST-HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {loadOnStartup=true, configSetProperties=configsetprops.json, config=solrconfig.xml, schema=schema.xml, name=mock, transient=false, dataDir=data/}
   [junit4]   2> 4175016 INFO  (TEST-HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:58558/solr
   [junit4]   2> 4175028 INFO  (TEST-HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 4175028 INFO  (TEST-HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [134217728] will allocate [1] slabs and use ~[134217728] bytes
   [junit4]   2> 4175028 INFO  (TEST-HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 4175613 ERROR (Finalizer) [    ] o.a.s.c.CoreContainer CoreContainer was not close prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!  instance=1802494744
   [junit4]   2> 4175614 ERROR (Finalizer) [    ] o.a.s.c.CoreContainer CoreContainer was not close prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!  instance=61693914
   [junit4]   2> 4175614 ERROR (Finalizer) [    ] o.a.s.c.CoreContainer CoreContainer was not close prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!  instance=1804671906
   [junit4]   2> 4175614 ERROR (Finalizer) [    ] o.a.s.c.CoreContainer CoreContainer was not close prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!  instance=705101462
   [junit4]   2> 4175708 INFO  (TEST-HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig-seed#[8F089C32D5F6338D]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testInitArgsOrSysPropConfig
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=HdfsDirectoryFactoryTest -Dtests.method=testInitArgsOrSysPropConfig -Dtests.seed=8F089C32D5F6338D -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=lt -Dtests.timezone=Europe/Belgrade -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.73s J2 | HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig <<<
   [junit4]    > Throwable #1: java.lang.RuntimeException: The max direct memory is likely too low.  Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~134.217728 MB.
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([8F089C32D5F6338D:78A75519287FD9A6]:0)
   [junit4]    > 	at org.apache.solr.core.HdfsDirectoryFactory.createBlockCache(HdfsDirectoryFactory.java:296)
   [junit4]    > 	at org.apache.solr.core.HdfsDirectoryFactory.getBlockDirectoryCache(HdfsDirectoryFactory.java:272)
   [junit4]    > 	at org.apache.solr.core.HdfsDirectoryFactory.create(HdfsDirectoryFactory.java:213)
   [junit4]    > 	at org.apache.solr.core.HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig(HdfsDirectoryFactoryTest.java:114)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]    > Caused by: java.lang.OutOfMemoryError: Direct buffer memory
   [junit4]    > 	at java.nio.Bits.reserveMemory(Bits.java:658)
   [junit4]    > 	at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
   [junit4]    > 	at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306)
   [junit4]    > 	at org.apache.solr.store.blockcache.BlockCache.<init>(BlockCache.java:64)
   [junit4]    > 	at org.apache.solr.core.HdfsDirectoryFactory.createBlockCache(HdfsDirectoryFactory.java:294)
   [junit4]    > 	... 42 more
   [junit4]   2> 4175712 WARN  (SUITE-HdfsDirectoryFactoryTest-seed#[8F089C32D5F6338D]-worker) [    ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
   [junit4]   2> 4175718 INFO  (SUITE-HdfsDirectoryFactoryTest-seed#[8F089C32D5F6338D]-worker) [    ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
   [junit4]   2> 4175721 WARN  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.HdfsDirectoryFactoryTest_8F089C32D5F6338D-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.HdfsDirectoryFactoryTest_8F089C32D5F6338D-001/tempDir-001/hdfsBaseDir/data/data4/]]  heartbeating to localhost/127.0.0.1:58558) [    ] o.a.h.h.s.d.DataNode BPOfferService for Block pool BP-1343184817-10.41.0.5-1452517860327 (Datanode Uuid 12802cc4-6db1-4ef5-8642-74a794f9f7e4) service to localhost/127.0.0.1:58558 interrupted
   [junit4]   2> 4175721 WARN  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.HdfsDirectoryFactoryTest_8F089C32D5F6338D-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.HdfsDirectoryFactoryTest_8F089C32D5F6338D-001/tempDir-001/hdfsBaseDir/data/data4/]]  heartbeating to localhost/127.0.0.1:58558) [    ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-1343184817-10.41.0.5-1452517860327 (Datanode Uuid 12802cc4-6db1-4ef5-8642-74a794f9f7e4) service to localhost/127.0.0.1:58558
   [junit4]   2> 4175724 WARN  (SUITE-HdfsDirectoryFactoryTest-seed#[8F089C32D5F6338D]-worker) [    ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
   [junit4]   2> 4175728 INFO  (SUITE-HdfsDirectoryFactoryTest-seed#[8F089C32D5F6338D]-worker) [    ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
   [junit4]   2> 4175830 WARN  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.HdfsDirectoryFactoryTest_8F089C32D5F6338D-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.HdfsDirectoryFactoryTest_8F089C32D5F6338D-001/tempDir-001/hdfsBaseDir/data/data2/]]  heartbeating to localhost/127.0.0.1:58558) [    ] o.a.h.h.s.d.DataNode BPOfferService for Block pool BP-1343184817-10.41.0.5-1452517860327 (Datanode Uuid 10416d2b-5638-40bc-a743-3ecf129c4740) service to localhost/127.0.0.1:58558 interrupted
   [junit4]   2> 4175830 WARN  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.HdfsDirectoryFactoryTest_8F089C32D5F6338D-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.HdfsDirectoryFactoryTest_8F089C32D5F6338D-001/tempDir-001/hdfsBaseDir/data/data2/]]  heartbeating to localhost/127.0.0.1:58558) [    ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-1343184817-10.41.0.5-1452517860327 (Datanode Uuid 10416d2b-5638-40bc-a743-3ecf129c4740) service to localhost/127.0.0.1:58558
   [junit4]   2> 4175838 WARN  (org.apache.hadoop.hdfs.server.blockmanagement.DecommissionManager$Monitor@9d3a9f1) [    ] o.a.h.h.s.b.DecommissionManager Monitor interrupted: java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 4175850 INFO  (SUITE-HdfsDirectoryFactoryTest-seed#[8F089C32D5F6338D]-worker) [    ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
   [junit4]   2> 4175952 ERROR (SUITE-HdfsDirectoryFactoryTest-seed#[8F089C32D5F6338D]-worker) [    ] o.a.h.m.l.MethodMetric Error invoking method getBlocksTotal
   [junit4]   2> java.lang.reflect.InvocationTargetException
   [junit4]   2> 	at sun.reflect.GeneratedMethodAccessor136.invoke(Unknown Source)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MethodMetric$2.snapshot(MethodMetric.java:111)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MethodMetric.snapshot(MethodMetric.java:144)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MetricsRegistry.snapshot(MetricsRegistry.java:387)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1.getMetrics(MetricsSourceBuilder.java:79)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:195)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:172)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:151)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getClassName(DefaultMBeanServerInterceptor.java:1804)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
   [junit4]   2> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
   [junit4]   2> 	at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:227)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:461)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:199)
   [junit4]   2> 	at org.apache.solr.core.HdfsDirectoryFactoryTest.teardownClass(HdfsDirectoryFactoryTest.java:67)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 4175956 INFO  (SUITE-HdfsDirectoryFactoryTest-seed#[8F089C32D5F6338D]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.HdfsDirectoryFactoryTest_8F089C32D5F6338D-001
   [junit4]   2> Jan 11, 2016 1:11:02 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=lt, timezone=Europe/Belgrade
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=2,free=395209656,total=520617984
   [junit4]   2> NOTE: All tests run in this JVM: [TestConfigSetImmutable, TestExtendedDismaxParser, TestPerFieldSimilarity, TestCharFilters, TestLeaderInitiatedRecoveryThread, TestRestoreCore, AsyncCallRequestStatusResponseTest, TestSolrQueryParser, StressHdfsTest, TestSolrIndexConfig, TestSearcherReuse, SolrTestCaseJ4Test, UnloadDistributedZkTest, PKIAuthenticationIntegrationTest, TestFieldSortValues, OverriddenZkACLAndCredentialsProvidersTest, RollingRestartTest, SpatialRPTFieldTypeTest, TestSuggestSpellingConverter, TestConfigReload, TestSolr4Spatial2, HLLSerializationTest, TestManagedSchemaFieldResource, TestSortingResponseWriter, AutoCommitTest, TestConfigSetsAPIZkFailure, TestRebalanceLeaders, TestMaxScoreQueryParser, TestSchemaVersionResource, TestMacros, QueryParsingTest, ClusterStateTest, AnalysisErrorHandlingTest, TestManagedSchema, TestGroupingSearch, RequestHandlersTest, TestSchemaNameResource, TestUseDocValuesAsStored2, CachingDirectoryFactoryTest, CoreAdminCreateDiscoverTest, DebugComponentTest, TestSolrJ, TestJmxIntegration, ShardSplitTest, TestShardHandlerFactory, DistanceUnitsTest, TestPostingsSolrHighlighter, StatelessScriptUpdateProcessorFactoryTest, LeaderElectionTest, TestReRankQParserPlugin, BinaryUpdateRequestHandlerTest, SortSpecParsingTest, CleanupOldIndexTest, SampleTest, DeleteReplicaTest, DistributedSuggestComponentTest, TestCollationField, TestCopyFieldCollectionResource, FieldMutatingUpdateProcessorTest, TestManagedSynonymFilterFactory, NoCacheHeaderTest, TestReversedWildcardFilterFactory, TestRuleBasedAuthorizationPlugin, SuggestComponentContextFilterQueryTest, TestSort, OverseerTest, CollectionStateFormat2Test, TestSolrXml, TestIndexSearcher, UpdateParamsTest, TestRecovery, TestSolrDeletionPolicy1, TestRequestStatusCollectionAPI, TestDefaultStatsCache, ScriptEngineTest, TestCollectionAPI, SuggesterWFSTTest, TestJmxMonitoredMap, TestReload, TestDownShardTolerantSearch, TestOverriddenPrefixQueryForCustomFieldType, SuggesterTSTTest, ZkStateWriterTest, TestFastWriter, TestXIncludeConfig, TestScoreJoinQPNoScore, TestSearchPerf, TestRestManager, TestFieldTypeCollectionResource, TestAuthenticationFramework, TriLevelCompositeIdRoutingTest, BasicDistributedZk2Test, HardAutoCommitTest, TestRangeQuery, SimpleFacetsTest, SolrCmdDistributorTest, QueryElevationComponentTest, PeerSyncTest, BadIndexSchemaTest, TestFiltering, BasicFunctionalityTest, ShowFileRequestHandlerTest, TestCoreDiscovery, CoreAdminHandlerTest, SpatialFilterTest, SchemaVersionSpecificBehaviorTest, TestPseudoReturnFields, TestUpdate, QueryEqualityTest, SortByFunctionTest, SolrInfoMBeanTest, LukeRequestHandlerTest, DirectSolrSpellCheckerTest, TestQueryTypes, TestOmitPositions, TestIndexingPerformance, FieldAnalysisRequestHandlerTest, LoggingHandlerTest, IndexSchemaTest, TestCSVResponseWriter, TestAnalyzedSuggestions, JsonLoaderTest, TestPHPSerializedResponseWriter, CSVRequestHandlerTest, SearchHandlerTest, TestLFUCache, PrimUtilsTest, SpellingQueryConverterTest, RAMDirectoryFactoryTest, TestUtils, ZkNodePropsTest, SliceStateTest, UUIDFieldTest, CircularListTest, DistributedMLTComponentTest, TestRTGBase, TestCrossCoreJoin, TestCursorMarkWithoutUniqueKey, TestDistributedMissingSort, TestSimpleTrackingShardHandler, TestTolerantSearch, TestEmbeddedSolrServerConstructors, ActionThrottleTest, AliasIntegrationTest, AssignTest, ConcurrentDeleteAndCreateCollectionTest, DeleteLastCustomShardedReplicaTest, DeleteShardTest, DistribDocExpirationUpdateProcessorTest, LeaderFailoverAfterPartitionTest, OutOfBoxZkACLAndCredentialsProvidersTest, OverseerCollectionConfigSetProcessorTest, OverseerRolesTest, OverseerTaskQueueTest, RemoteQueryErrorTest, SSLMigrationTest, SaslZkACLProviderTest, SharedFSAutoReplicaFailoverUtilsTest, SimpleCollectionCreateDeleteTest, TestDistribDocBasedVersion, TestExclusionRuleCollectionAccess, TestMiniSolrCloudCluster, TestSolrCloudWithKerberosAlt, HdfsBasicDistributedZk2Test, HdfsChaosMonkeyNothingIsSafeTest, HdfsCollectionsAPIDistributedZkTest, HdfsNNFailoverTest, HdfsRecoverLeaseTest, HdfsRecoveryZkTest, HdfsThreadLeakTest, HdfsUnloadDistributedZkTest, HdfsWriteToMultipleCollectionsTest, DirectoryFactoryTest, HdfsDirectoryFactoryTest]
   [junit4] Completed [392/563 (1!)] on J2 in 12.24s, 3 tests, 1 error <<< FAILURES!

[...truncated 141 lines...]
   [junit4] Suite: org.apache.solr.index.hdfs.CheckHdfsIndexTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.index.hdfs.CheckHdfsIndexTest_8F089C32D5F6338D-001/init-core-data-001
   [junit4]   2> 4342768 INFO  (SUITE-CheckHdfsIndexTest-seed#[8F089C32D5F6338D]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 4342768 INFO  (SUITE-CheckHdfsIndexTest-seed#[8F089C32D5F6338D]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 4342823 WARN  (SUITE-CheckHdfsIndexTest-seed#[8F089C32D5F6338D]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 4342833 WARN  (SUITE-CheckHdfsIndexTest-seed#[8F089C32D5F6338D]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 4342836 INFO  (SUITE-CheckHdfsIndexTest-seed#[8F089C32D5F6338D]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 4342851 INFO  (SUITE-CheckHdfsIndexTest-seed#[8F089C32D5F6338D]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_46627_hdfs____igpckj/webapp
   [junit4]   2> 4342969 INFO  (SUITE-CheckHdfsIndexTest-seed#[8F089C32D5F6338D]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 4343225 INFO  (SUITE-CheckHdfsIndexTest-seed#[8F089C32D5F6338D]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46627
   [junit4]   2> 4343287 WARN  (SUITE-CheckHdfsIndexTest-seed#[8F089C32D5F6338D]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 4343288 INFO  (SUITE-CheckHdfsIndexTest-seed#[8F089C32D5F6338D]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 4343301 INFO  (SUITE-CheckHdfsIndexTest-seed#[8F089C32D5F6338D]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_37565_datanode____.i40y15/webapp
   [junit4]   2> 4343420 INFO  (SUITE-CheckHdfsIndexTest-seed#[8F089C32D5F6338D]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 4343747 INFO  (SUITE-CheckHdfsIndexTest-seed#[8F089C32D5F6338D]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37565
   [junit4]   2> 4343788 WARN  (SUITE-CheckHdfsIndexTest-seed#[8F089C32D5F6338D]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 4343790 INFO  (SUITE-CheckHdfsIndexTest-seed#[8F089C32D5F6338D]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 4343808 INFO  (SUITE-CheckHdfsIndexTest-seed#[8F089C32D5F6338D]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_57628_datanode____.k7473/webapp
   [junit4]   2> 4343861 INFO  (IPC Server handler 3 on 45277) [    ] BlockStateChange BLOCK* processReport: from storage DS-5425b123-006c-4d62-b8b5-e8ad6be4168d node DatanodeRegistration(127.0.0.1, datanodeUuid=53147476-ab76-433d-b821-7ffed9dbe7d8, infoPort=37565, ipcPort=42432, storageInfo=lv=-56;cid=testClusterID;nsid=1800155769;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 4343861 INFO  (IPC Server handler 3 on 45277) [    ] BlockStateChange BLOCK* processReport: from storage DS-4909e491-591a-42fa-b72c-1c8e79000a33 node DatanodeRegistration(127.0.0.1, datanodeUuid=53147476-ab76-433d-b821-7ffed9dbe7d8, infoPort=37565, ipcPort=42432, storageInfo=lv=-56;cid=testClusterID;nsid=1800155769;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 4343953 INFO  (SUITE-CheckHdfsIndexTest-seed#[8F089C32D5F6338D]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 4344172 INFO  (SUITE-CheckHdfsIndexTest-seed#[8F089C32D5F6338D]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:57628
   [junit4]   2> 4344254 INFO  (IPC Server handler 8 on 45277) [    ] BlockStateChange BLOCK* processReport: from storage DS-eac235cc-8b4b-4730-b5aa-ab6a40b2f47d node DatanodeRegistration(127.0.0.1, datanodeUuid=8985b884-e001-4a1e-8937-e60e78f5219f, infoPort=57628, ipcPort=55836, storageInfo=lv=-56;cid=testClusterID;nsid=1800155769;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 4344254 INFO  (IPC Server handler 8 on 45277) [    ] BlockStateChange BLOCK* processReport: from storage DS-79cde9ca-ce66-420b-8081-f9c1d37b0741 node DatanodeRegistration(127.0.0.1, datanodeUuid=8985b884-e001-4a1e-8937-e60e78f5219f, infoPort=57628, ipcPort=55836, storageInfo=lv=-56;cid=testClusterID;nsid=1800155769;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 4344386 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 4344386 INFO  (Thread-237313) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 4344386 INFO  (Thread-237313) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 4344486 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.ZkTestServer start zk server on port:40186
   [junit4]   2> 4344487 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4344487 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4344490 INFO  (zkCallback-2515-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@bfbb183 name:ZooKeeperConnection Watcher:127.0.0.1:40186 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4344490 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4344491 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4344491 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 4344494 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4344494 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4344496 INFO  (zkCallback-2516-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5b315586 name:ZooKeeperConnection Watcher:127.0.0.1:40186/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4344496 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4344496 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4344496 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 4344498 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 4344500 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 4344501 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 4344503 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 4344503 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 4344506 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 4344506 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 4344508 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 4344508 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 4344509 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 4344510 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 4344511 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 4344511 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 4344513 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 4344513 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 4344515 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 4344515 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 4344516 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 4344517 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 4344518 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 4344518 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 4344520 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 4344520 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 4344522 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 4344522 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 4344819 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.index.hdfs.CheckHdfsIndexTest_8F089C32D5F6338D-001/control-001/cores/collection1
   [junit4]   2> 4344821 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 4344823 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4603914c{/,null,AVAILABLE}
   [junit4]   2> 4344823 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.e.j.s.ServerConnector Started ServerConnector@7704a0e9{HTTP/1.1}{127.0.0.1:54596}
   [junit4]   2> 4344823 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.e.j.s.Server Started @4347668ms
   [junit4]   2> 4344823 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, hostPort=54596, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.index.hdfs.CheckHdfsIndexTest_8F089C32D5F6338D-001/control-001/cores, solr.data.dir=hdfs://localhost:45277/hdfs__localhost_45277__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J0_temp_solr.index.hdfs.CheckHdfsIndexTest_8F089C32D5F6338D-001_tempDir-002_control_data}
   [junit4]   2> 4344823 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 4344824 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.index.hdfs.CheckHdfsIndexTest_8F089C32D5F6338D-001/control-001'
   [junit4]   2> 4344824 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 4344824 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 4344824 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4344825 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4344826 INFO  (zkCallback-2517-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1226be15 name:ZooKeeperConnection Watcher:127.0.0.1:40186/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4344826 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4344827 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4344828 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 4344828 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.index.hdfs.CheckHdfsIndexTest_8F089C32D5F6338D-001/control-001/solr.xml
   [junit4]   2> 4344842 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.index.hdfs.CheckHdfsIndexTest_8F089C32D5F6338D-001/control-001/cores
   [junit4]   2> 4344843 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.CoreContainer New CoreContainer 553885970
   [junit4]   2> 4344843 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.index.hdfs.CheckHdfsIndexTest_8F089C32D5F6338D-001/control-001]
   [junit4]   2> 4344843 WARN  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.index.hdfs.CheckHdfsIndexTest_8F089C32D5F6338D-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.index.hdfs.CheckHdfsIndexTest_8F089C32D5F6338D-001/control-001/lib
   [junit4]   2> 4344843 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 4344844 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 4344845 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 4344845 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 4344845 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40186/solr
   [junit4]   2> 4344845 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 4344845 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4344846 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4344847 INFO  (zkCallback-2519-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@32835e5f name:ZooKeeperConnection Watcher:127.0.0.1:40186 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4344847 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4344847 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4344849 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [n:127.0.0.1:54596_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4344850 INFO  (zkCallback-2520-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@24f6cedf name:ZooKeeperConnection Watcher:127.0.0.1:40186/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4344850 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [n:127.0.0.1:54596_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4344851 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [n:127.0.0.1:54596_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 4344853 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [n:127.0.0.1:54596_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 4344855 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [n:127.0.0.1:54596_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 4344857 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [n:127.0.0.1:54596_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 4344858 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [n:127.0.0.1:54596_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 4344860 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [n:127.0.0.1:54596_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 4344861 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [n:127.0.0.1:54596_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 4344863 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [n:127.0.0.1:54596_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 4344864 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [n:127.0.0.1:54596_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 4344864 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [n:127.0.0.1:54596_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54596_
   [junit4]   2> 4344865 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [n:127.0.0.1:54596_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:54596_
   [junit4]   2> 4344866 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [n:127.0.0.1:54596_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 4344867 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [n:127.0.0.1:54596_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 4344868 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [n:127.0.0.1:54596_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 4344869 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [n:127.0.0.1:54596_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95192221676732420-127.0.0.1:54596_-n_0000000000
   [junit4]   2> 4344869 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [n:127.0.0.1:54596_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:54596_
   [junit4]   2> 4344869 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [n:127.0.0.1:54596_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 4344871 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [n:127.0.0.1:54596_    ] o.a.s.c.Overseer Overseer (id=95192221676732420-127.0.0.1:54596_-n_0000000000) starting
   [junit4]   2> 4344872 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [n:127.0.0.1:54596_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 4344877 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [n:127.0.0.1:54596_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 4344878 INFO  (OverseerCollectionConfigSetProcessor-95192221676732420-127.0.0.1:54596_-n_0000000000) [n:127.0.0.1:54596_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 4344878 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [n:127.0.0.1:54596_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 4344878 INFO  (OverseerStateUpdate-95192221676732420-127.0.0.1:54596_-n_0000000000) [n:127.0.0.1:54596_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 4344896 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [n:127.0.0.1:54596_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 4344896 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [n:127.0.0.1:54596_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 4344897 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [n:127.0.0.1:54596_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.index.hdfs.CheckHdfsIndexTest_8F089C32D5F6338D-001/control-001/cores
   [junit4]   2> 4344898 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [n:127.0.0.1:54596_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {collection=control_collection, coreNodeName=, loadOnStartup=true, config=solrconfig.xml, shard=, schema=schema.xml, configSetProperties=configsetprops.json, dataDir=data/, name=collection1, transient=false}
   [junit4]   2> 4344898 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [n:127.0.0.1:54596_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.index.hdfs.CheckHdfsIndexTest_8F089C32D5F6338D-001/control-001/cores/collection1
   [junit4]   2> 4344898 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [n:127.0.0.1:54596_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 4344899 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [n:127.0.0.1:54596_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 4344900 INFO  (TEST-CheckHdfsIndexTest.testChecksumsOnly-seed#[8F089C32D5F6338D]) [n:127.0.0.1:54596_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 4344900 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 4344900 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 4344902 INFO  (OverseerStateUpdate-95192221676732420-127.0.0.1:54596_-n_0000000000) [n:127.0.0.1:54596_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:54596_",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:54596",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down"} current state version: 0
   [junit4]   2> 4344903 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 4344903 INFO  (OverseerStateUpdate-95192221676732420-127.0.0.1:54596_-n_0000000000) [n:127.0.0.1:54596_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "node_name":"127.0.0.1:54596_",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:54596",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down"}
   [junit4]   2> 4344903 INFO  (OverseerStateUpdate-95192221676732420-127.0.0.1:54596_-n_0000000000) [n:127.0.0.1:54596_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 4344904 INFO  (OverseerStateUpdate-95192221676732420-127.0.0.1:54596_-n_0000000000) [n:127.0.0.1:54596_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 4344905 INFO  (zkCallback-2520-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 4345903 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 4345903 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 4345904 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 4345904 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 4345905 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 4345905 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.index.hdfs.CheckHdfsIndexTest_8F089C32D5F6338D-001/control-001/cores/collection1'
   [junit4]   2> 4345906 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 4345906 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 4345909 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 4345914 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 4345928 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 4345956 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 4345964 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 4346226 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 4346234 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 4346236 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 4346245 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 4346248 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 4346251 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 4346252 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 4346253 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 4346253 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 4346254 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 4346254 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:45277/solr_hdfs_home
   [junit4]   2> 4346254 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 4346254 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.index.hdfs.CheckHdfsIndexTest_8F089C32D5F6338D-001/control-001/cores/collection1], dataDir=[hdfs://localhost:45277/solr_hdfs_home/control_collection/core_node1/data/]
   [junit4]   2> 4346254 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5a872a2d
   [junit4]   2> 4346255 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:45277/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 4346271 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:45277/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 4346272 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:45277/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 4346282 WARN  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:45277/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 4346282 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:45277/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 4346291 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 4346291 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 4346295 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 4346295 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:45277/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 4346295 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=5, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 4346310 INFO  (IPC Server handler 4 on 45277) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:55884 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-4909e491-591a-42fa-b72c-1c8e79000a33:NORMAL:127.0.0.1:51362|RBW], ReplicaUnderConstruction[[DISK]DS-79cde9ca-ce66-420b-8081-f9c1d37b0741:NORMAL:127.0.0.1:55884|FINALIZED]]} size 0
   [junit4]   2> 4346311 INFO  (IPC Server handler 5 on 45277) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:51362 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-79cde9ca-ce66-420b-8081-f9c1d37b0741:NORMAL:127.0.0.1:55884|FINALIZED], ReplicaUnderConstruction[[DISK]DS-5425b123-006c-4d62-b8b5-e8ad6be4168d:NORMAL:127.0.0.1:51362|FINALIZED]]} size 0
   [junit4]   2> 4346314 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@hdfs://localhost:45277/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@672f5e74); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 4346314 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 4346325 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:54596_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:45277/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 4346328 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 4346328 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 4346328 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 4346329 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 4346329 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 4346329 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 4346329 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 4346329 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 4346329 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 4346329 WARN  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 4346332 INFO  (coreLoadExecutor-8237-thread-1-processing-n:127.0.0.1:54596_) [n:127.0.0.1:54596_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xs

[...truncated too long message...]

/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-admin-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.2.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20160111_120123_535.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20160111_120123_535.suites -stdin
   [junit4] ERROR: JVM J1 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1444: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1000: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 230 minutes 54 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1071 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1071/

1 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=11471, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=11471, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:44248/a: collection already exists: awholynewstresscollection_collection1_0
	at __randomizedtesting.SeedInfo.seed([6084C11E6ED1C557]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:881)




Build Log:
[...truncated 10967 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/init-core-data-001
   [junit4]   2> 1148256 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[6084C11E6ED1C557]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true)
   [junit4]   2> 1148256 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[6084C11E6ED1C557]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /a/
   [junit4]   2> 1148260 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1148260 INFO  (Thread-6464) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1148260 INFO  (Thread-6464) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1148360 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.ZkTestServer start zk server on port:58605
   [junit4]   2> 1148360 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1148361 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1148364 INFO  (zkCallback-1014-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7af29c70 name:ZooKeeperConnection Watcher:127.0.0.1:58605 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1148364 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1148364 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1148364 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1148367 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1148367 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1148368 INFO  (zkCallback-1015-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@fb58e9a name:ZooKeeperConnection Watcher:127.0.0.1:58605/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1148368 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1148369 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1148369 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1148370 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1148372 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1148373 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1148375 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1148375 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1148377 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1148377 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1148379 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1148379 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1148381 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1148381 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1148383 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1148383 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1148384 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1148384 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1148386 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1148386 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1148388 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1148388 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1148390 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1148390 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1148392 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1148392 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1148394 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1148394 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1148396 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1148397 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1148398 INFO  (zkCallback-1016-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7a50967f name:ZooKeeperConnection Watcher:127.0.0.1:58605/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1148398 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1148398 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1148701 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/control-001/cores/collection1
   [junit4]   2> 1148703 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1148705 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2d18227a{/a,null,AVAILABLE}
   [junit4]   2> 1148711 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.e.j.s.ServerConnector Started ServerConnector@671a7f2f{SSL-http/1.1}{127.0.0.1:38201}
   [junit4]   2> 1148711 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.e.j.s.Server Started @1151657ms
   [junit4]   2> 1148712 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/tempDir-001/control/data, hostContext=/a, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/control-001/cores, hostPort=38201}
   [junit4]   2> 1148712 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1148712 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/control-001'
   [junit4]   2> 1148712 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1148712 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1148712 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1148713 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1148714 INFO  (zkCallback-1017-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1795fff0 name:ZooKeeperConnection Watcher:127.0.0.1:58605/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1148714 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1148715 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1148715 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1148716 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/control-001/solr.xml
   [junit4]   2> 1148735 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/control-001/cores
   [junit4]   2> 1148735 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.CoreContainer New CoreContainer 1632270101
   [junit4]   2> 1148735 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/control-001]
   [junit4]   2> 1148735 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/control-001/lib
   [junit4]   2> 1148736 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1148740 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1148742 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1148742 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1148742 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:58605/solr
   [junit4]   2> 1148743 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1148743 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1148743 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1148745 INFO  (zkCallback-1019-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@58dfbc6c name:ZooKeeperConnection Watcher:127.0.0.1:58605 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1148745 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1148746 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1148747 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:38201_a    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1148749 INFO  (zkCallback-1020-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@c574eae name:ZooKeeperConnection Watcher:127.0.0.1:58605/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1148749 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:38201_a    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1148750 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:38201_a    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1148752 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:38201_a    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1148756 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:38201_a    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1148761 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:38201_a    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1148763 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:38201_a    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1148766 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:38201_a    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1148768 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:38201_a    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1148770 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:38201_a    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1148771 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:38201_a    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1148772 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:38201_a    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38201_a
   [junit4]   2> 1148781 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:38201_a    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:38201_a
   [junit4]   2> 1148783 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:38201_a    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1148784 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:38201_a    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1148785 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:38201_a    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1148785 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:38201_a    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95186531110289413-127.0.0.1:38201_a-n_0000000000
   [junit4]   2> 1148786 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:38201_a    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:38201_a
   [junit4]   2> 1148786 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:38201_a    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1148787 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:38201_a    ] o.a.s.c.Overseer Overseer (id=95186531110289413-127.0.0.1:38201_a-n_0000000000) starting
   [junit4]   2> 1148788 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:38201_a    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1148793 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:38201_a    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1148798 INFO  (OverseerCollectionConfigSetProcessor-95186531110289413-127.0.0.1:38201_a-n_0000000000) [n:127.0.0.1:38201_a    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1148800 INFO  (OverseerStateUpdate-95186531110289413-127.0.0.1:38201_a-n_0000000000) [n:127.0.0.1:38201_a    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1148802 INFO  (OverseerStateUpdate-95186531110289413-127.0.0.1:38201_a-n_0000000000) [n:127.0.0.1:38201_a    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1148807 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:38201_a    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1148846 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:38201_a    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1148846 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:38201_a    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1148847 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:38201_a    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/control-001/cores
   [junit4]   2> 1148849 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:38201_a    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {coreNodeName=, loadOnStartup=true, collection=control_collection, schema=schema.xml, name=collection1, dataDir=data/, configSetProperties=configsetprops.json, shard=, config=solrconfig.xml, transient=false}
   [junit4]   2> 1148849 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:38201_a    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/control-001/cores/collection1
   [junit4]   2> 1148849 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:38201_a    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1148851 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:38201_a    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1148851 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:38201_a    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1148851 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1148851 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1148854 INFO  (OverseerStateUpdate-95186531110289413-127.0.0.1:38201_a-n_0000000000) [n:127.0.0.1:38201_a    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"https://127.0.0.1:38201/a",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:38201_a"} current state version: 0
   [junit4]   2> 1148854 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1148855 INFO  (OverseerStateUpdate-95186531110289413-127.0.0.1:38201_a-n_0000000000) [n:127.0.0.1:38201_a    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"https://127.0.0.1:38201/a",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:38201_a"}
   [junit4]   2> 1148855 INFO  (OverseerStateUpdate-95186531110289413-127.0.0.1:38201_a-n_0000000000) [n:127.0.0.1:38201_a    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1148856 INFO  (OverseerStateUpdate-95186531110289413-127.0.0.1:38201_a-n_0000000000) [n:127.0.0.1:38201_a    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1148865 INFO  (zkCallback-1020-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1148865 INFO  (zkCallback-1020-thread-2-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1149855 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1149855 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1149855 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1149855 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 1149856 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1149856 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/control-001/cores/collection1'
   [junit4]   2> 1149856 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1149856 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1149858 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1149864 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1149878 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1149907 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1149915 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1150187 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1150197 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1150199 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1150211 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1150215 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1150218 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1150218 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1150220 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1150220 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1150220 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1150221 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/control-001/cores/collection1/data/]
   [junit4]   2> 1150221 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@65cb8923
   [junit4]   2> 1150222 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/control-001/cores/collection1/data
   [junit4]   2> 1150222 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1150222 WARN  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1150222 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/control-001/cores/collection1/data/index
   [junit4]   2> 1150223 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=7.609375, floorSegmentMB=0.6083984375, forceMergeDeletesPctAllowed=15.059272419023873, segmentsPerTier=40.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7281672057648912
   [junit4]   2> 1150223 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@27451c71 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2a4e52db),segFN=segments_1,generation=1}
   [junit4]   2> 1150223 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1150224 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/control-001/cores/collection1/data/
   [junit4]   2> 1150225 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1150236 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1150237 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1150237 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1150237 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1150237 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1150237 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1150237 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1150238 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1150238 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1150238 WARN  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 1150241 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1150243 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1150245 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1150246 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1150251 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/segments,/dump,/update/json,/replication,/admin/logging,/get,/admin/ping,/update/json/docs,standard,/config,/admin/plugins,/admin/luke,/admin/system,/admin/file,/admin/mbeans,/admin/threads,/update,/schema,/update/csv,/admin/properties
   [junit4]   2> 1150253 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1150253 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1150253 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1150254 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1150254 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1150255 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=10, maxMergeAtOnceExplicit=48, maxMergedSegmentMB=0.2900390625, floorSegmentMB=2.150390625, forceMergeDeletesPctAllowed=17.50575100611905, segmentsPerTier=32.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5502066819220731
   [junit4]   2> 1150255 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@27451c71 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2a4e52db),segFN=segments_1,generation=1}
   [junit4]   2> 1150255 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1150256 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@7fbe124f[collection1] main
   [junit4]   2> 1150256 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 1150257 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1150257 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1150257 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1150257 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1150257 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1150258 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1150258 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1150258 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1150258 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1150259 INFO  (searcherExecutor-2700-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7fbe124f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1150259 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1150259 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1150259 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1150259 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1522984499858636800
   [junit4]   2> 1150261 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1522984499858636800
   [junit4]   2> 1150261 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1150262 INFO  (coreLoadExecutor-2699-thread-1-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1150262 INFO  (coreZkRegister-2693-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:https://127.0.0.1:38201/a collection:control_collection shard:shard1
   [junit4]   2> 1150263 INFO  (coreZkRegister-2693-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1150267 INFO  (coreZkRegister-2693-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95186531110289413-core_node1-n_0000000000
   [junit4]   2> 1150267 INFO  (coreZkRegister-2693-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=180000
   [junit4]   2> 1150269 INFO  (coreZkRegister-2693-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1150270 INFO  (OverseerStateUpdate-95186531110289413-127.0.0.1:38201_a-n_0000000000) [n:127.0.0.1:38201_a    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1150270 INFO  (coreZkRegister-2693-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1150271 INFO  (coreZkRegister-2693-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:38201/a/collection1/
   [junit4]   2> 1150271 INFO  (coreZkRegister-2693-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1150271 INFO  (coreZkRegister-2693-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:38201/a/collection1/ has no replicas
   [junit4]   2> 1150272 INFO  (coreZkRegister-2693-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1150276 INFO  (coreZkRegister-2693-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1150278 INFO  (coreZkRegister-2693-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:38201/a/collection1/ shard1
   [junit4]   2> 1150279 INFO  (OverseerStateUpdate-95186531110289413-127.0.0.1:38201_a-n_0000000000) [n:127.0.0.1:38201_a    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"https://127.0.0.1:38201/a",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1150355 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1150355 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1150356 INFO  (zkCallback-1022-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@17e3423c name:ZooKeeperConnection Watcher:127.0.0.1:58605/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1150357 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1150357 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1150357 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1150360 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1150360 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 1150360 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1150360 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1150361 INFO  (zkCallback-1023-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@41928fef name:ZooKeeperConnection Watcher:127.0.0.1:58605/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1150361 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1150362 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1150364 INFO  (OverseerStateUpdate-95186531110289413-127.0.0.1:38201_a-n_0000000000) [n:127.0.0.1:38201_a    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 1
   [junit4]   2> 1150364 INFO  (OverseerStateUpdate-95186531110289413-127.0.0.1:38201_a-n_0000000000) [n:127.0.0.1:38201_a    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1150365 INFO  (zkCallback-1022-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1150365 INFO  (zkCallback-1020-thread-3-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1150365 INFO  (zkCallback-1020-thread-2-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1150379 INFO  (coreZkRegister-2693-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are https://127.0.0.1:38201/a/collection1/ and leader is https://127.0.0.1:38201/a/collection1/
   [junit4]   2> 1150380 INFO  (coreZkRegister-2693-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:38201/a
   [junit4]   2> 1150380 INFO  (coreZkRegister-2693-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1150380 INFO  (coreZkRegister-2693-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1150380 INFO  (coreZkRegister-2693-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1150381 INFO  (OverseerStateUpdate-95186531110289413-127.0.0.1:38201_a-n_0000000000) [n:127.0.0.1:38201_a    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"https://127.0.0.1:38201/a",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "node_name":"127.0.0.1:38201_a"} current state version: 2
   [junit4]   2> 1150382 INFO  (OverseerStateUpdate-95186531110289413-127.0.0.1:38201_a-n_0000000000) [n:127.0.0.1:38201_a    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"https://127.0.0.1:38201/a",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "node_name":"127.0.0.1:38201_a"}
   [junit4]   2> 1150383 INFO  (OverseerStateUpdate-95186531110289413-127.0.0.1:38201_a-n_0000000000) [n:127.0.0.1:38201_a    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 1150486 INFO  (zkCallback-1022-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1150486 INFO  (zkCallback-1020-thread-2-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1150486 INFO  (zkCallback-1020-thread-3-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1150722 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/shard-1-001/cores/collection1
   [junit4]   2> 1150722 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/shard-1-001
   [junit4]   2> 1150723 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1150733 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2787e0b3{/a,null,AVAILABLE}
   [junit4]   2> 1150738 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.e.j.s.ServerConnector Started ServerConnector@aa83bf9{SSL-http/1.1}{127.0.0.1:49206}
   [junit4]   2> 1150738 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.e.j.s.Server Started @1153684ms
   [junit4]   2> 1150738 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/a, hostPort=49206, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/tempDir-001/jetty1, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/shard-1-001/cores, solrconfig=solrconfig.xml}
   [junit4]   2> 1150738 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1150738 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/shard-1-001'
   [junit4]   2> 1150738 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1150739 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1150739 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1150740 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1150741 INFO  (zkCallback-1024-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@53d46f3 name:ZooKeeperConnection Watcher:127.0.0.1:58605/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1150742 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1150742 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1150743 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1150743 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/shard-1-001/solr.xml
   [junit4]   2> 1150757 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/shard-1-001/cores
   [junit4]   2> 1150757 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.CoreContainer New CoreContainer 543568153
   [junit4]   2> 1150757 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/shard-1-001]
   [junit4]   2> 1150758 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/shard-1-001/lib
   [junit4]   2> 1150758 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1150760 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1150762 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1150762 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1150762 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:58605/solr
   [junit4]   2> 1150762 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1150762 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1150762 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1150765 INFO  (zkCallback-1026-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4b90a433 name:ZooKeeperConnection Watcher:127.0.0.1:58605 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1150766 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1150766 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1150768 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:49206_a    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1150771 INFO  (zkCallback-1027-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6495f01 name:ZooKeeperConnection Watcher:127.0.0.1:58605/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1150771 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:49206_a    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1150776 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:49206_a    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1151781 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:49206_a    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49206_a
   [junit4]   2> 1151782 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:49206_a    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:49206_a
   [junit4]   2> 1151783 INFO  (zkCallback-1027-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1151783 INFO  (zkCallback-1022-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1151783 INFO  (zkCallback-1020-thread-2-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1151783 INFO  (zkCallback-1020-thread-3-processing-n:127.0.0.1:38201_a) [n:127.0.0.1:38201_a    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1151785 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:49206_a    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1151786 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:49206_a    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95186531110289418-127.0.0.1:49206_a-n_0000000001
   [junit4]   2> 1151787 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:49206_a    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/95186531110289413-127.0.0.1:38201_a-n_0000000000 to know if I could be the leader
   [junit4]   2> 1151810 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:49206_a    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1151810 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:49206_a    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1151811 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:49206_a    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/shard-1-001/cores
   [junit4]   2> 1151812 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:49206_a    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {config=solrconfig.xml, dataDir=data/, transient=false, collection=collection1, name=collection1, loadOnStartup=true, coreNodeName=, schema=schema.xml, shard=, configSetProperties=configsetprops.json}
   [junit4]   2> 1151812 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:49206_a    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/shard-1-001/cores/collection1
   [junit4]   2> 1151812 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:49206_a    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1151814 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:49206_a    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1151815 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6084C11E6ED1C557]) [n:127.0.0.1:49206_a    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1151817 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1151817 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1151819 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 1151819 INFO  (OverseerStateUpdate-95186531110289413-127.0.0.1:38201_a-n_0000000000) [n:127.0.0.1:38201_a    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"https://127.0.0.1:49206/a",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "node_name":"127.0.0.1:49206_a"} current state version: 3
   [junit4]   2> 1151819 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch [collection1]
   [junit4]   2> 1151819 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for [collection1] ver [0]
   [junit4]   2> 1151820 INFO  (OverseerStateUpdate-95186531110289413-127.0.0.1:38201_a-n_0000000000) [n:127.0.0.1:38201_a    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"https://127.0.0.1:49206/a",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "node_name":"127.0.0.1:49206_a"}
   [junit4]   2> 1151820 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1151820 INFO  (OverseerStateUpdate-95186531110289413-127.0.0.1:38201_a-n_0000000000) [n:127.0.0.1:38201_a    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=2
   [junit4]   2> 1151820 INFO  (OverseerStateUpdate-95186531110289413-127.0.0.1:38201_a-n_0000000000) [n:127.0.0.1:38201_a    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 1151925 INFO  (OverseerStateUpdate-95186531110289413-127.0.0.1:38201_a-n_0000000000) [n:127.0.0.1:38201_a    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 1151926 INFO  (zkCallback-1027-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1151926 INFO  (zkCallback-1027-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a    ] o.a.s.c.c.ZkStateReader Updating data for [collection1] from [0] to [1]
   [junit4]   2> 1152820 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1152820 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1152820 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1152820 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/collection1]
   [junit4]   2> 1152821 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/collection1] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1152821 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/shard-1-001/cores/collection1'
   [junit4]   2> 1152821 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1152821 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1152823 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1152829 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1152842 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1152869 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1152877 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1153125 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1153133 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1153135 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1153144 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1153147 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1153149 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1153150 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1153152 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1153152 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 1153152 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1153152 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/shard-1-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/shard-1-001/cores/collection1/data/]
   [junit4]   2> 1153153 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@65cb8923
   [junit4]   2> 1153153 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/shard-1-001/cores/collection1/data
   [junit4]   2> 1153153 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/shard-1-001/cores/collection1/data/index/
   [junit4]   2> 1153153 WARN  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/shard-1-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1153154 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_6084C11E6ED1C557-001/shard-1-001/cores/collection1/data/index
   [junit4]   2> 1153154 INFO  (coreLoadExecutor-2710-thread-1-processing-n:127.0.0.1:49206_a) [n:127.0.0.1:49206_a c:collection1 s:shard2 r:core_node1 x:collection1] o.a.

[...truncated too long message...]

junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:881)Throwable #4: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=11474, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:44248/a: collection already exists: awholynewstresscollection_collection4_0
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([6084C11E6ED1C557]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:881)Throwable #5: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=11475, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:44248/a: collection already exists: awholynewstresscollection_collection5_0
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([6084C11E6ED1C557]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:881)Throwable #6: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=11473, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:33363/a: Could not find collection : awholynewstresscollection_collection3_0
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([6084C11E6ED1C557]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)
   [junit4]   2> 1687035 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[6084C11E6ED1C557]-worker) [n:127.0.0.1:44248_a c:addReplicaColl s:shard2 r:core_node3 x:addReplicaColl_shard2_replica2] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=es_UY, timezone=Europe/Busingen
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=138675704,total=508559360
   [junit4]   2> NOTE: All tests run in this JVM: [DistributedQueryComponentCustomSortTest, ExplicitHLLTest, IndexSchemaRuntimeFieldTest, TestHashQParserPlugin, TestBulkSchemaAPI, OutputWriterTest, CursorPagingTest, TestAnalyzedSuggestions, TestSimpleQParserPlugin, UUIDFieldTest, ReturnFieldsTest, CopyFieldTest, AtomicUpdatesTest, DefaultValueUpdateProcessorTest, TestHdfsUpdateLog, OverseerTaskQueueTest, SSLMigrationTest, DisMaxRequestHandlerTest, TestFieldCollectionResource, TestComponentsName, TestFastOutputStream, TestBulkSchemaConcurrent, TestScoreJoinQPScore, TestAuthorizationFramework, IndexSchemaTest, TestUniqueKeyFieldResource, TestLRUCache, TestAnalyzeInfixSuggestions, DistributedExpandComponentTest, XsltUpdateRequestHandlerTest, HdfsChaosMonkeyNothingIsSafeTest, TestDefaultSearchFieldResource, TestMiniSolrCloudClusterSSL, CollectionTooManyReplicasTest, DistribJoinFromCollectionTest, OverseerCollectionConfigSetProcessorTest, TestRangeQuery, TestStressLucene, DateFieldTest, SecurityConfHandlerTest, PeerSyncTest, TestCSVLoader, VMParamsZkACLAndCredentialsProvidersTest, FileBasedSpellCheckerTest, PingRequestHandlerTest, TestLRUStatsCache, TestConfigSetProperties, DocumentAnalysisRequestHandlerTest, SuggestComponentTest, TestFoldingMultitermQuery, TestReplicationHandlerBackup, DistributedQueryElevationComponentTest, TestSchemaManager, SolrCoreTest, StandardRequestHandlerTest, PreAnalyzedFieldTest, ResourceLoaderTest, TestBadConfig, TestChildDocTransformer, QueryResultKeyTest, HighlighterMaxOffsetTest, DocValuesMissingTest, TestManagedSchemaFieldTypeResource, TestPseudoReturnFields, TestValueSourceCache, TestRandomCollapseQParserPlugin, SignatureUpdateProcessorFactoryTest, TestAddFieldRealTimeGet, TestIBSimilarityFactory, TestMergePolicyConfig, TestLMDirichletSimilarityFactory, SimpleMLTQParserTest, HdfsNNFailoverTest, TestClusterStateMutator, URLClassifyProcessorTest, TestSerializedLuceneMatchVersion, PolyFieldTest, BigEndianAscendingWordDeserializerTest, TestCloudSchemaless, PathHierarchyTokenizerFactoryTest, TestMiniSolrCloudCluster, FacetPivotSmallTest, ForceLeaderTest, TestSchemaResource, HLLUtilTest, TestBinaryField, TestCryptoKeys, TestCrossCoreJoin, TestFieldTypeResource, TestPhraseSuggestions, BasicZkTest, TestArbitraryIndexDir, CurrencyFieldOpenExchangeTest, CollectionsAPIDistributedZkTest]
   [junit4] Completed [271/563 (1!)] on J0 in 540.43s, 1 test, 1 error <<< FAILURES!

[...truncated 967 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1444: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1000: There were test failures: 563 suites (7 ignored), 2236 tests, 1 error, 33 ignored (18 assumptions) [seed: 6084C11E6ED1C557]

Total time: 201 minutes 14 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1070 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1070/

1 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=12228, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=12228, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
	at __randomizedtesting.SeedInfo.seed([F8D786BCDB755BB6:7083B9667589364E]:0)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:52410/js_aay/yg: Could not find collection : awholynewstresscollection_collection3_0
	at __randomizedtesting.SeedInfo.seed([F8D786BCDB755BB6]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)




Build Log:
[...truncated 11043 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_F8D786BCDB755BB6-001/init-core-data-001
   [junit4]   2> 1500555 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[F8D786BCDB755BB6]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1500555 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[F8D786BCDB755BB6]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /js_aay/yg
   [junit4]   2> 1500560 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1500560 INFO  (Thread-6877) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1500560 INFO  (Thread-6877) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1500660 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.ZkTestServer start zk server on port:50082
   [junit4]   2> 1500661 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1500664 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1500673 INFO  (zkCallback-978-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@23e4e606 name:ZooKeeperConnection Watcher:127.0.0.1:50082 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1500674 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1500674 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1500674 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1500717 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1500723 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1500725 INFO  (zkCallback-979-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@79b87229 name:ZooKeeperConnection Watcher:127.0.0.1:50082/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1500725 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1500725 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1500726 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1500728 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1500729 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1500730 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1500731 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1500731 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1500733 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1500734 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1500735 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1500735 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1500737 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1500737 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1500738 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1500738 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1500740 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1500740 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1500741 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1500741 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1500742 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1500743 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1500744 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1500744 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1500746 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1500746 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1500747 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1500747 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1500750 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1500751 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1500752 INFO  (zkCallback-980-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@fff237c name:ZooKeeperConnection Watcher:127.0.0.1:50082 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1500752 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1500752 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1500752 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1500755 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1500760 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1500761 INFO  (zkCallback-981-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@795d0ae3 name:ZooKeeperConnection Watcher:127.0.0.1:50082/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1500761 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1500761 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1500762 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 1500762 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 1500764 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 1500764 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 1500767 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 1500767 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 1500768 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1500769 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1500770 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 1500770 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 1500771 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 1500771 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 1500773 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 1500773 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 1500774 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1500774 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1500776 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1500776 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1500777 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 1500777 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 1500779 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 1500779 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 1500780 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 1500781 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 1501287 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_F8D786BCDB755BB6-001/control-001/cores/collection1
   [junit4]   2> 1501295 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1501305 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@493832b8{/js_aay/yg,null,AVAILABLE}
   [junit4]   2> 1501306 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.e.j.s.ServerConnector Started ServerConnector@889f396{HTTP/1.1}{127.0.0.1:40998}
   [junit4]   2> 1501306 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.e.j.s.Server Started @1504508ms
   [junit4]   2> 1501306 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_F8D786BCDB755BB6-001/tempDir-001/control/data, hostContext=/js_aay/yg, hostPort=40998, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_F8D786BCDB755BB6-001/control-001/cores}
   [junit4]   2> 1501306 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1501306 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_F8D786BCDB755BB6-001/control-001'
   [junit4]   2> 1501306 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1501307 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1501307 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1501317 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1501325 INFO  (zkCallback-982-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@73dc03d5 name:ZooKeeperConnection Watcher:127.0.0.1:50082/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1501325 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1501325 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1501328 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1501328 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_F8D786BCDB755BB6-001/control-001/solr.xml
   [junit4]   2> 1501350 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_F8D786BCDB755BB6-001/control-001/cores
   [junit4]   2> 1501350 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.CoreContainer New CoreContainer 172824435
   [junit4]   2> 1501350 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_F8D786BCDB755BB6-001/control-001]
   [junit4]   2> 1501351 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_F8D786BCDB755BB6-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_F8D786BCDB755BB6-001/control-001/lib
   [junit4]   2> 1501351 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1501352 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1501353 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1501353 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1501353 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:50082/solr
   [junit4]   2> 1501353 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1501353 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1501354 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1501360 INFO  (zkCallback-984-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@50f71415 name:ZooKeeperConnection Watcher:127.0.0.1:50082 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1501361 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1501361 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1501363 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1501368 INFO  (zkCallback-985-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@27f645b9 name:ZooKeeperConnection Watcher:127.0.0.1:50082/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1501368 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1501372 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1501377 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1501381 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1501385 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1501388 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1501391 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1501393 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1501394 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1501396 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1501397 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40998_js_aay%2Fyg
   [junit4]   2> 1501398 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:40998_js_aay%2Fyg
   [junit4]   2> 1501405 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1501408 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1501409 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1501410 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95180949146566662-127.0.0.1:40998_js_aay%2Fyg-n_0000000000
   [junit4]   2> 1501410 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:40998_js_aay%2Fyg
   [junit4]   2> 1501410 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1501411 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.Overseer Overseer (id=95180949146566662-127.0.0.1:40998_js_aay%2Fyg-n_0000000000) starting
   [junit4]   2> 1501413 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1501420 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1501429 INFO  (OverseerStateUpdate-95180949146566662-127.0.0.1:40998_js_aay%2Fyg-n_0000000000) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1501429 INFO  (OverseerStateUpdate-95180949146566662-127.0.0.1:40998_js_aay%2Fyg-n_0000000000) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1501436 INFO  (OverseerCollectionConfigSetProcessor-95180949146566662-127.0.0.1:40998_js_aay%2Fyg-n_0000000000) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1501452 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1501497 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1501497 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1501499 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_F8D786BCDB755BB6-001/control-001/cores
   [junit4]   2> 1501499 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {transient=false, shard=, dataDir=data/, coreNodeName=, name=collection1, config=solrconfig.xml, schema=schema.xml, collection=control_collection, configSetProperties=configsetprops.json, loadOnStartup=true}
   [junit4]   2> 1501500 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_F8D786BCDB755BB6-001/control-001/cores/collection1
   [junit4]   2> 1501500 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1501515 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1501515 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1501516 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1501516 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1501522 INFO  (OverseerStateUpdate-95180949146566662-127.0.0.1:40998_js_aay%2Fyg-n_0000000000) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:40998_js_aay%2Fyg",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:40998/js_aay/yg",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1501523 INFO  (OverseerStateUpdate-95180949146566662-127.0.0.1:40998_js_aay%2Fyg-n_0000000000) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "node_name":"127.0.0.1:40998_js_aay%2Fyg",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:40998/js_aay/yg",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1501524 INFO  (OverseerStateUpdate-95180949146566662-127.0.0.1:40998_js_aay%2Fyg-n_0000000000) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1501524 INFO  (OverseerStateUpdate-95180949146566662-127.0.0.1:40998_js_aay%2Fyg-n_0000000000) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1501527 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1501528 INFO  (zkCallback-985-thread-2-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1501528 INFO  (zkCallback-985-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1502528 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1502528 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1502529 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1502529 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 1502529 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1502529 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_F8D786BCDB755BB6-001/control-001/cores/collection1'
   [junit4]   2> 1502530 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1502530 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1502533 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1502552 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1502571 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1502619 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1502630 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1502944 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1502962 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1502963 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1502985 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1502990 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1502994 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1502995 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1502997 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1502997 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1502997 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1502997 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_F8D786BCDB755BB6-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_F8D786BCDB755BB6-001/control-001/cores/collection1/data/]
   [junit4]   2> 1502997 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@80efb76
   [junit4]   2> 1502998 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_F8D786BCDB755BB6-001/control-001/cores/collection1/data
   [junit4]   2> 1502998 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_F8D786BCDB755BB6-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1502998 WARN  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_F8D786BCDB755BB6-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1502999 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_F8D786BCDB755BB6-001/control-001/cores/collection1/data/index
   [junit4]   2> 1502999 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=3, maxMergeAtOnceExplicit=9, maxMergedSegmentMB=88.99609375, floorSegmentMB=2.107421875, forceMergeDeletesPctAllowed=6.338324463624896, segmentsPerTier=27.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1502999 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@198f797a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1a7045b2),segFN=segments_1,generation=1}
   [junit4]   2> 1503000 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1503000 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_F8D786BCDB755BB6-001/control-001/cores/collection1/data/
   [junit4]   2> 1503001 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_F8D786BCDB755BB6-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1503007 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1503008 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1503008 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1503008 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1503008 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1503008 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1503009 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1503009 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1503009 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1503009 WARN  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 1503013 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1503014 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1503016 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1503017 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1503023 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/plugins,/admin/ping,/admin/file,/dump,standard,/admin/system,/admin/mbeans,/admin/properties,/admin/luke,/admin/logging,/get,/schema,/update/csv,/replication,/config,/update,/admin/segments,/admin/threads,/update/json/docs,/update/json
   [junit4]   2> 1503024 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1503025 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1503025 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1503026 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1503026 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1503026 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@5e92ed4b
   [junit4]   2> 1503027 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@198f797a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1a7045b2),segFN=segments_1,generation=1}
   [junit4]   2> 1503027 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1503027 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@56a1ca94[collection1] main
   [junit4]   2> 1503027 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 1503034 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1503034 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1503034 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1503034 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1503034 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1503035 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1503035 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1503035 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1503035 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1503036 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1503036 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1503036 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1503036 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1522895188939243520
   [junit4]   2> 1503038 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1522895188939243520
   [junit4]   2> 1503039 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1503040 INFO  (searcherExecutor-3414-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@56a1ca94[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1503044 INFO  (coreLoadExecutor-3413-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1503044 INFO  (coreZkRegister-3407-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:40998/js_aay/yg collection:control_collection shard:shard1
   [junit4]   2> 1503050 INFO  (coreZkRegister-3407-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1503054 INFO  (coreZkRegister-3407-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95180949146566662-core_node1-n_0000000000
   [junit4]   2> 1503054 INFO  (coreZkRegister-3407-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=180000
   [junit4]   2> 1503055 INFO  (coreZkRegister-3407-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1503056 INFO  (coreZkRegister-3407-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1503056 INFO  (coreZkRegister-3407-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40998/js_aay/yg/collection1/
   [junit4]   2> 1503056 INFO  (coreZkRegister-3407-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1503056 INFO  (coreZkRegister-3407-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:40998/js_aay/yg/collection1/ has no replicas
   [junit4]   2> 1503057 INFO  (OverseerStateUpdate-95180949146566662-127.0.0.1:40998_js_aay%2Fyg-n_0000000000) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1503058 INFO  (zkCallback-985-thread-2-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1503058 INFO  (zkCallback-985-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1503059 INFO  (coreZkRegister-3407-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1503068 INFO  (coreZkRegister-3407-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1503070 INFO  (coreZkRegister-3407-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40998/js_aay/yg/collection1/ shard1
   [junit4]   2> 1503073 INFO  (OverseerStateUpdate-95180949146566662-127.0.0.1:40998_js_aay%2Fyg-n_0000000000) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:40998/js_aay/yg",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 1503182 INFO  (zkCallback-985-thread-2-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1503182 INFO  (zkCallback-985-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1503225 INFO  (coreZkRegister-3407-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:40998/js_aay/yg/collection1/ and leader is http://127.0.0.1:40998/js_aay/yg/collection1/
   [junit4]   2> 1503225 INFO  (coreZkRegister-3407-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:40998/js_aay/yg
   [junit4]   2> 1503225 INFO  (coreZkRegister-3407-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1503225 INFO  (coreZkRegister-3407-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1503225 INFO  (coreZkRegister-3407-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:40998_js_aay%2Fyg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1503227 INFO  (OverseerStateUpdate-95180949146566662-127.0.0.1:40998_js_aay%2Fyg-n_0000000000) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "node_name":"127.0.0.1:40998_js_aay%2Fyg",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:40998/js_aay/yg",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 1503227 INFO  (OverseerStateUpdate-95180949146566662-127.0.0.1:40998_js_aay%2Fyg-n_0000000000) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "node_name":"127.0.0.1:40998_js_aay%2Fyg",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:40998/js_aay/yg",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1503334 INFO  (zkCallback-985-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1503334 INFO  (zkCallback-985-thread-2-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1503518 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1503519 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1503521 INFO  (zkCallback-987-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4c5d822e name:ZooKeeperConnection Watcher:127.0.0.1:50082/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1503522 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1503522 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1503522 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1503534 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1503534 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 1503535 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1503537 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1503539 INFO  (zkCallback-988-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@21539420 name:ZooKeeperConnection Watcher:127.0.0.1:50082/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1503540 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1503540 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1503542 INFO  (OverseerStateUpdate-95180949146566662-127.0.0.1:40998_js_aay%2Fyg-n_0000000000) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 4
   [junit4]   2> 1503542 INFO  (OverseerStateUpdate-95180949146566662-127.0.0.1:40998_js_aay%2Fyg-n_0000000000) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1503646 INFO  (OverseerStateUpdate-95180949146566662-127.0.0.1:40998_js_aay%2Fyg-n_0000000000) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 1504047 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_F8D786BCDB755BB6-001/shard-1-001/cores/collection1
   [junit4]   2> 1504048 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_F8D786BCDB755BB6-001/shard-1-001
   [junit4]   2> 1504049 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1504057 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@23c68066{/js_aay/yg,null,AVAILABLE}
   [junit4]   2> 1504057 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.e.j.s.ServerConnector Started ServerConnector@3d17bae3{HTTP/1.1}{127.0.0.1:58588}
   [junit4]   2> 1504057 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.e.j.s.Server Started @1507259ms
   [junit4]   2> 1504057 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/js_aay/yg, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_F8D786BCDB755BB6-001/tempDir-001/jetty1, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_F8D786BCDB755BB6-001/shard-1-001/cores, hostPort=58588, solrconfig=solrconfig.xml}
   [junit4]   2> 1504058 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1504058 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_F8D786BCDB755BB6-001/shard-1-001'
   [junit4]   2> 1504058 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1504058 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1504058 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1504065 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1504074 INFO  (zkCallback-989-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2844c8dc name:ZooKeeperConnection Watcher:127.0.0.1:50082/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1504074 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1504075 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1504076 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1504076 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_F8D786BCDB755BB6-001/shard-1-001/solr.xml
   [junit4]   2> 1504102 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_F8D786BCDB755BB6-001/shard-1-001/cores
   [junit4]   2> 1504103 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.CoreContainer New CoreContainer 358927931
   [junit4]   2> 1504103 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_F8D786BCDB755BB6-001/shard-1-001]
   [junit4]   2> 1504103 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_F8D786BCDB755BB6-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_F8D786BCDB755BB6-001/shard-1-001/lib
   [junit4]   2> 1504111 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1504113 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1504114 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1504114 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1504114 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:50082/solr
   [junit4]   2> 1504114 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1504114 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1504122 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1504146 INFO  (zkCallback-991-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@38d7efff name:ZooKeeperConnection Watcher:127.0.0.1:50082 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1504147 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1504148 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1504158 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:58588_js_aay%2Fyg    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1504167 INFO  (zkCallback-992-thread-1-processing-n:127.0.0.1:58588_js_aay%2Fyg) [n:127.0.0.1:58588_js_aay%2Fyg    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@267c1b76 name:ZooKeeperConnection Watcher:127.0.0.1:50082/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1504167 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:58588_js_aay%2Fyg    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1504172 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:58588_js_aay%2Fyg    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1505183 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:58588_js_aay%2Fyg    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58588_js_aay%2Fyg
   [junit4]   2> 1505183 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:58588_js_aay%2Fyg    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:58588_js_aay%2Fyg
   [junit4]   2> 1505184 INFO  (zkCallback-987-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1505184 INFO  (zkCallback-985-thread-2-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1505185 INFO  (zkCallback-985-thread-1-processing-n:127.0.0.1:40998_js_aay%2Fyg) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1505185 INFO  (zkCallback-992-thread-1-processing-n:127.0.0.1:58588_js_aay%2Fyg) [n:127.0.0.1:58588_js_aay%2Fyg    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1505187 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:58588_js_aay%2Fyg    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1505188 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:58588_js_aay%2Fyg    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95180949146566667-127.0.0.1:58588_js_aay%2Fyg-n_0000000001
   [junit4]   2> 1505189 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:58588_js_aay%2Fyg    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/95180949146566662-127.0.0.1:40998_js_aay%2Fyg-n_0000000000 to know if I could be the leader
   [junit4]   2> 1505208 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:58588_js_aay%2Fyg    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1505209 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:58588_js_aay%2Fyg    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1505209 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:58588_js_aay%2Fyg    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_F8D786BCDB755BB6-001/shard-1-001/cores
   [junit4]   2> 1505210 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:58588_js_aay%2Fyg    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {transient=false, configSetProperties=configsetprops.json, coreNodeName=, collection=collection1, dataDir=data/, name=collection1, shard=, schema=schema.xml, loadOnStartup=true, config=solrconfig.xml}
   [junit4]   2> 1505211 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:58588_js_aay%2Fyg    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_F8D786BCDB755BB6-001/shard-1-001/cores/collection1
   [junit4]   2> 1505211 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:58588_js_aay%2Fyg    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1505212 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:58588_js_aay%2Fyg    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1505212 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[F8D786BCDB755BB6]) [n:127.0.0.1:58588_js_aay%2Fyg    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1505212 INFO  (coreLoadExecutor-3424-thread-1-processing-n:127.0.0.1:58588_js_aay%2Fyg) [n:127.0.0.1:58588_js_aay%2Fyg c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1505212 INFO  (coreLoadExecutor-3424-thread-1-processing-n:127.0.0.1:58588_js_aay%2Fyg) [n:127.0.0.1:58588_js_aay%2Fyg c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1505213 INFO  (coreLoadExecutor-3424-thread-1-processing-n:127.0.0.1:58588_js_aay%2Fyg) [n:127.0.0.1:58588_js_aay%2Fyg c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 1505214 INFO  (coreLoadExecutor-3424-thread-1-processing-n:127.0.0.1:58588_js_aay%2Fyg) [n:127.0.0.1:58588_js_aay%2Fyg c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch [collection1]
   [junit4]   2> 1505214 INFO  (OverseerStateUpdate-95180949146566662-127.0.0.1:40998_js_aay%2Fyg-n_0000000000) [n:127.0.0.1:40998_js_aay%2Fyg    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:58588_js_aay%2Fyg",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:58588/js_aay/yg",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state"} current state version: 4
   [junit4]   2> 1505214 INFO  (coreLoadExecutor-3424-thread-1-processing-n:127.0.0.1:58588_js_aay%2Fyg) [n:127.0.0.1:58588_js_aay%2Fy

[...truncated too long message...]

t4]   2> 	2	/solr/collections/awholynewstresscollection_collection2_3/leader_elect/shard2/election/95180949146566662-core_node7-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection2_0/leader_elect/shard2/election/95180949146566683-core_node5-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection4_0/leader_elect/shard4/election/95180949146566676-core_node10-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection3_0/leader_elect/shard4/election/95180949146566670-core_node15-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection5_3/leader_elect/shard5/election/95180949146566683-core_node14-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection0_3/leader_elect/shard6/election/95180949146566670-core_node4-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection5_3/leader_elect/shard1/election/95180949146566667-core_node1-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection3_0/leader_elect/shard2/election/95180949146566683-core_node6-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection3_0/leader_elect/shard6/election/95180949146566667-core_node17-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection3_0/leader_elect/shard5/election/95180949146566676-core_node14-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection2_0/leader_elect/shard6/election/95180949146566667-core_node10-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection2_3/leader_elect/shard1/election/95180949146566662-core_node2-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection2_0/leader_elect/shard1/election/95180949146566676-core_node13-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection4_3/leader_elect/shard1/election/95180949146566667-core_node5-n_0000000001
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	88	/solr/collections
   [junit4]   2> 	30	/solr/live_nodes
   [junit4]   2> 	2	/solr/overseer/queue
   [junit4]   2> 	2	/solr/overseer/collection-queue-work
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=CollectionsAPIDistributedZkTest -Dtests.method=test -Dtests.seed=F8D786BCDB755BB6 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=de_LU -Dtests.timezone=America/Paramaribo -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR    280s J0 | CollectionsAPIDistributedZkTest.test <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=12228, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F8D786BCDB755BB6:7083B9667589364E]:0)
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:52410/js_aay/yg: Could not find collection : awholynewstresscollection_collection3_0
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F8D786BCDB755BB6]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)
   [junit4]   2> 1780904 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[F8D786BCDB755BB6]-worker) [n:127.0.0.1:52692_js_aay%2Fyg c:addReplicaColl s:shard1 r:core_node7 x:propertyDotName] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Jan 09, 2016 1:31:46 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {range_facet_l=PostingsFormat(name=Memory doPackFST= true), other_tl1=PostingsFormat(name=Memory doPackFST= true), a_t=PostingsFormat(name=Memory doPackFST= true), text=Lucene50(blocksize=128), id=PostingsFormat(name=LuceneVarGapFixedInterval), multiDefault=BlockTreeOrds(blocksize=128), intDefault=PostingsFormat(name=Memory doPackFST= true), timestamp=PostingsFormat(name=Memory doPackFST= true), a_i1=PostingsFormat(name=Memory doPackFST= true), _version_=PostingsFormat(name=Memory doPackFST= true), range_facet_i_dv=PostingsFormat(name=Memory doPackFST= true), range_facet_l_dv=PostingsFormat(name=LuceneVarGapFixedInterval)}, docValues:{timestamp=DocValuesFormat(name=Direct), range_facet_i_dv=DocValuesFormat(name=Direct), range_facet_l_dv=DocValuesFormat(name=Asserting)}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=de_LU, timezone=America/Paramaribo
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=99494544,total=483393536
   [junit4]   2> NOTE: All tests run in this JVM: [TestSchemaResource, HdfsUnloadDistributedZkTest, URLClassifyProcessorTest, TestChildDocTransformer, ExplicitHLLTest, TestRecoveryHdfs, TestNamedUpdateProcessors, SmileWriterTest, IndexSchemaTest, TestLuceneMatchVersion, UUIDFieldTest, RuleEngineTest, TestReplicaProperties, PathHierarchyTokenizerFactoryTest, TestScoreJoinQPScore, TestLRUCache, TestFastOutputStream, BlockJoinFacetRandomTest, TestRTGBase, SolrCloudExampleTest, HLLUtilTest, TestComplexPhraseQParserPlugin, TestRandomMergePolicy, TestRealTimeGet, QueryElevationComponentTest, TestSolrConfigHandler, DocValuesMissingTest, AtomicUpdatesTest, HdfsLockFactoryTest, TestEmbeddedSolrServerConstructors, CloudExitableDirectoryReaderTest, DistributedQueryComponentCustomSortTest, JsonLoaderTest, AnalyticsMergeStrategyTest, HighlighterMaxOffsetTest, AnalysisAfterCoreReloadTest, TestHighlightDedupGrouping, ForceLeaderTest, TestAnalyzedSuggestions, TestFaceting, ReplicationFactorTest, TestRandomFaceting, TestCSVLoader, OverseerCollectionConfigSetProcessorTest, BadCopyFieldTest, TestReplicationHandlerBackup, OverseerTaskQueueTest, TestManagedResource, TestPivotHelperCode, TestUniqueKeyFieldResource, TestIBSimilarityFactory, PeerSyncTest, FileBasedSpellCheckerTest, RequestLoggingTest, TestPhraseSuggestions, CollectionTooManyReplicasTest, TestStandardQParsers, LeaderElectionIntegrationTest, TestHashQParserPlugin, CoreAdminRequestStatusTest, TestRemoteStreaming, SSLMigrationTest, TestFoldingMultitermQuery, OutputWriterTest, TestConfigSetProperties, IndexSchemaRuntimeFieldTest, SynonymTokenizerTest, HdfsChaosMonkeyNothingIsSafeTest, JSONWriterTest, XsltUpdateRequestHandlerTest, LeaderInitiatedRecoveryOnCommitTest, TestConfig, TestPerFieldSimilarityWithDefaultOverride, TestDefaultSearchFieldResource, TestDocSet, TestPseudoReturnFields, TestSortByMinMaxFunction, IgnoreCommitOptimizeUpdateProcessorFactoryTest, HdfsNNFailoverTest, SolrRequestParserTest, SuggestComponentTest, SharedFSAutoReplicaFailoverTest, DistributedFacetPivotSmallAdvancedTest, RAMDirectoryFactoryTest, TestCrossCoreJoin, TestCloudInspectUtil, DistribJoinFromCollectionTest, TestAddFieldRealTimeGet, TestRawTransformer, PrimUtilsTest, PingRequestHandlerTest, TestJsonFacets, TestConfigSetsAPI, DeleteShardTest, BasicDistributedZkTest, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, CollectionsAPIDistributedZkTest]
   [junit4] Completed [287/563 (1!)] on J0 in 281.43s, 1 test, 1 error <<< FAILURES!

[...truncated 908 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1444: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1000: There were test failures: 563 suites (7 ignored), 2236 tests, 1 error, 33 ignored (18 assumptions) [seed: F8D786BCDB755BB6]

Total time: 203 minutes 18 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1069 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1069/

6 tests failed.
FAILED:  org.apache.solr.cloud.BasicDistributedZk2Test.test

Error Message:
Doc Counts do not add up expected:<67> but was:<1>

Stack Trace:
java.lang.AssertionError: Doc Counts do not add up expected:<67> but was:<1>
	at __randomizedtesting.SeedInfo.seed([74AB2677F5DB2089:FCFF19AD5B274D71]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.assertDocCounts(AbstractFullDistribZkTestBase.java:1368)
	at org.apache.solr.cloud.BasicDistributedZk2Test.test(BasicDistributedZk2Test.java:91)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:965)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:940)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test

Error Message:
expected:<1> but was:<0>

Stack Trace:
java.lang.AssertionError: expected:<1> but was:<0>
	at __randomizedtesting.SeedInfo.seed([74AB2677F5DB2089:FCFF19AD5B274D71]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.junit.Assert.assertEquals(Assert.java:456)
	at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.addUpdateDelete(FullSolrCloudDistribCmdsTest.java:480)
	at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test(FullSolrCloudDistribCmdsTest.java:82)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:965)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:940)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.ShardSplitTest.test

Error Message:
There are still nodes recoverying - waited for 330 seconds

Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
	at __randomizedtesting.SeedInfo.seed([74AB2677F5DB2089:FCFF19AD5B274D71]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:175)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:136)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:831)
	at org.apache.solr.cloud.ShardSplitTest.splitByRouteFieldTest(ShardSplitTest.java:292)
	at org.apache.solr.cloud.ShardSplitTest.test(ShardSplitTest.java:83)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:965)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:940)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.TestAuthenticationFramework.testBasics

Error Message:
expected:<1> but was:<0>

Stack Trace:
java.lang.AssertionError: expected:<1> but was:<0>
	at __randomizedtesting.SeedInfo.seed([74AB2677F5DB2089:4973885BCD357EF9]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.junit.Assert.assertEquals(Assert.java:456)
	at org.apache.solr.cloud.TestMiniSolrCloudCluster.testCollectionCreateSearchDelete(TestMiniSolrCloudCluster.java:175)
	at org.apache.solr.cloud.TestAuthenticationFramework.testBasics(TestAuthenticationFramework.java:93)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest.test

Error Message:
shard1 is not consistent.  Got 2078 from http://127.0.0.1:38168/collection1 (previous client) and got 2117 from http://127.0.0.1:36992/collection1

Stack Trace:
java.lang.AssertionError: shard1 is not consistent.  Got 2078 from http://127.0.0.1:38168/collection1 (previous client) and got 2117 from http://127.0.0.1:36992/collection1
	at __randomizedtesting.SeedInfo.seed([74AB2677F5DB2089:FCFF19AD5B274D71]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1246)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1225)
	at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test(ChaosMonkeySafeLeaderTest.java:175)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:965)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:940)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.StressHdfsTest.test

Error Message:
Error from server at http://127.0.0.1:58914: Could not find collection : delete_data_dir

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:58914: Could not find collection : delete_data_dir
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.hdfs.StressHdfsTest.createAndDeleteCollection(StressHdfsTest.java:206)
	at org.apache.solr.cloud.hdfs.StressHdfsTest.test(StressHdfsTest.java:100)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:965)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:940)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10250 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_74AB2677F5DB2089-001/init-core-data-001
   [junit4]   2> 186696 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[74AB2677F5DB2089]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 186696 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[74AB2677F5DB2089]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 186845 WARN  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[74AB2677F5DB2089]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 186857 WARN  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[74AB2677F5DB2089]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 186861 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[74AB2677F5DB2089]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 186902 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[74AB2677F5DB2089]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_43701_hdfs____q4qbsz/webapp
   [junit4]   2> 187127 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[74AB2677F5DB2089]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 187562 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[74AB2677F5DB2089]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43701
   [junit4]   2> 187623 WARN  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[74AB2677F5DB2089]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 187625 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[74AB2677F5DB2089]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 187637 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[74AB2677F5DB2089]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_54082_datanode____.3tb8mm/webapp
   [junit4]   2> 187751 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[74AB2677F5DB2089]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 188016 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[74AB2677F5DB2089]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:54082
   [junit4]   2> 188079 WARN  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[74AB2677F5DB2089]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 188082 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[74AB2677F5DB2089]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 188097 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[74AB2677F5DB2089]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_54623_datanode____.m00e1t/webapp
   [junit4]   2> 188187 INFO  (IPC Server handler 3 on 47484) [    ] BlockStateChange BLOCK* processReport: from storage DS-8ed71e16-9545-47f1-943f-b509f583feeb node DatanodeRegistration(127.0.0.1, datanodeUuid=1b2d57f2-6f83-44ca-912b-43eb689b21b6, infoPort=54082, ipcPort=48076, storageInfo=lv=-56;cid=testClusterID;nsid=1005100663;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 188187 INFO  (IPC Server handler 3 on 47484) [    ] BlockStateChange BLOCK* processReport: from storage DS-2953dc3f-3138-45d5-be78-06abf999a567 node DatanodeRegistration(127.0.0.1, datanodeUuid=1b2d57f2-6f83-44ca-912b-43eb689b21b6, infoPort=54082, ipcPort=48076, storageInfo=lv=-56;cid=testClusterID;nsid=1005100663;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 188232 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[74AB2677F5DB2089]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 188734 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[74AB2677F5DB2089]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:54623
   [junit4]   2> 188858 INFO  (IPC Server handler 4 on 47484) [    ] BlockStateChange BLOCK* processReport: from storage DS-9c6bef0f-f1a9-45cb-92b5-815dbd5f4a03 node DatanodeRegistration(127.0.0.1, datanodeUuid=9fc89753-2044-404d-9368-8f3163abfb5a, infoPort=54623, ipcPort=46672, storageInfo=lv=-56;cid=testClusterID;nsid=1005100663;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 188858 INFO  (IPC Server handler 4 on 47484) [    ] BlockStateChange BLOCK* processReport: from storage DS-152bd8a2-f747-4d7d-9ef2-41e2a136814d node DatanodeRegistration(127.0.0.1, datanodeUuid=9fc89753-2044-404d-9368-8f3163abfb5a, infoPort=54623, ipcPort=46672, storageInfo=lv=-56;cid=testClusterID;nsid=1005100663;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 188903 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 188903 INFO  (Thread-5135) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 188903 INFO  (Thread-5135) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 189004 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.ZkTestServer start zk server on port:35223
   [junit4]   2> 189004 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 189005 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 189008 INFO  (zkCallback-97-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@713f719e name:ZooKeeperConnection Watcher:127.0.0.1:35223 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 189008 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 189008 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 189008 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 189011 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 189012 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 189013 INFO  (zkCallback-98-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@699583e0 name:ZooKeeperConnection Watcher:127.0.0.1:35223/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 189013 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 189013 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 189014 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 189017 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 189019 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 189021 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 189024 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 189024 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 189028 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 189028 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 189031 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 189031 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 189034 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 189034 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 189036 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 189036 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 189039 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 189039 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 189042 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 189042 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 189044 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 189044 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 189047 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 189047 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 189050 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 189050 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 189052 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 189053 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 189338 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_74AB2677F5DB2089-001/control-001/cores/collection1
   [junit4]   2> 189340 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 189342 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2d7bfef1{/,null,AVAILABLE}
   [junit4]   2> 189343 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.e.j.s.ServerConnector Started ServerConnector@1cef3aff{HTTP/1.1}{127.0.0.1:54347}
   [junit4]   2> 189343 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.e.j.s.Server Started @192143ms
   [junit4]   2> 189343 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_74AB2677F5DB2089-001/control-001/cores, hostPort=54347, solr.data.dir=hdfs://localhost:47484/hdfs__localhost_47484__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_74AB2677F5DB2089-001_tempDir-002_control_data, hostContext=/}
   [junit4]   2> 189343 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 189343 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_74AB2677F5DB2089-001/control-001'
   [junit4]   2> 189343 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 189343 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 189344 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 189344 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 189346 INFO  (zkCallback-99-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@133e4f84 name:ZooKeeperConnection Watcher:127.0.0.1:35223/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 189346 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 189346 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 189348 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 189348 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_74AB2677F5DB2089-001/control-001/solr.xml
   [junit4]   2> 189364 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_74AB2677F5DB2089-001/control-001/cores
   [junit4]   2> 189364 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.CoreContainer New CoreContainer 2084757845
   [junit4]   2> 189364 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_74AB2677F5DB2089-001/control-001]
   [junit4]   2> 189364 WARN  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_74AB2677F5DB2089-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_74AB2677F5DB2089-001/control-001/lib
   [junit4]   2> 189365 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 189366 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 189366 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 189366 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 189367 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35223/solr
   [junit4]   2> 189380 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 189380 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 189381 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 189382 INFO  (zkCallback-101-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@c5ea741 name:ZooKeeperConnection Watcher:127.0.0.1:35223 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 189383 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 189383 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 189388 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [n:127.0.0.1:54347_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 189390 INFO  (zkCallback-102-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@50bab4bf name:ZooKeeperConnection Watcher:127.0.0.1:35223/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 189390 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [n:127.0.0.1:54347_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 189397 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [n:127.0.0.1:54347_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 189404 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [n:127.0.0.1:54347_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 189409 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [n:127.0.0.1:54347_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 189413 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [n:127.0.0.1:54347_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 189415 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [n:127.0.0.1:54347_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 189429 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [n:127.0.0.1:54347_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 189431 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [n:127.0.0.1:54347_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 189433 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [n:127.0.0.1:54347_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 189435 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [n:127.0.0.1:54347_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 189436 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [n:127.0.0.1:54347_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54347_
   [junit4]   2> 189438 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [n:127.0.0.1:54347_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:54347_
   [junit4]   2> 189443 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [n:127.0.0.1:54347_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 189445 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [n:127.0.0.1:54347_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 189447 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [n:127.0.0.1:54347_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 189448 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [n:127.0.0.1:54347_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95173549299400708-127.0.0.1:54347_-n_0000000000
   [junit4]   2> 189452 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [n:127.0.0.1:54347_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:54347_
   [junit4]   2> 189452 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [n:127.0.0.1:54347_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 189454 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [n:127.0.0.1:54347_    ] o.a.s.c.Overseer Overseer (id=95173549299400708-127.0.0.1:54347_-n_0000000000) starting
   [junit4]   2> 189460 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [n:127.0.0.1:54347_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 189489 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [n:127.0.0.1:54347_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 189491 INFO  (OverseerCollectionConfigSetProcessor-95173549299400708-127.0.0.1:54347_-n_0000000000) [n:127.0.0.1:54347_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 189491 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [n:127.0.0.1:54347_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 189492 INFO  (OverseerStateUpdate-95173549299400708-127.0.0.1:54347_-n_0000000000) [n:127.0.0.1:54347_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 189607 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [n:127.0.0.1:54347_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 189607 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [n:127.0.0.1:54347_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 189609 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [n:127.0.0.1:54347_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_74AB2677F5DB2089-001/control-001/cores
   [junit4]   2> 189611 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [n:127.0.0.1:54347_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {shard=, configSetProperties=configsetprops.json, collection=control_collection, loadOnStartup=true, coreNodeName=, schema=schema.xml, name=collection1, transient=false, config=solrconfig.xml, dataDir=data/}
   [junit4]   2> 189611 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [n:127.0.0.1:54347_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_74AB2677F5DB2089-001/control-001/cores/collection1
   [junit4]   2> 189611 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [n:127.0.0.1:54347_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 189613 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 189616 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 189619 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [n:127.0.0.1:54347_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 189619 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[74AB2677F5DB2089]) [n:127.0.0.1:54347_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 189623 INFO  (OverseerStateUpdate-95173549299400708-127.0.0.1:54347_-n_0000000000) [n:127.0.0.1:54347_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:54347",
   [junit4]   2>   "node_name":"127.0.0.1:54347_",
   [junit4]   2>   "state":"down"} current state version: 0
   [junit4]   2> 189624 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 189633 INFO  (OverseerStateUpdate-95173549299400708-127.0.0.1:54347_-n_0000000000) [n:127.0.0.1:54347_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:54347",
   [junit4]   2>   "node_name":"127.0.0.1:54347_",
   [junit4]   2>   "state":"down"}
   [junit4]   2> 189635 INFO  (OverseerStateUpdate-95173549299400708-127.0.0.1:54347_-n_0000000000) [n:127.0.0.1:54347_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 189645 INFO  (OverseerStateUpdate-95173549299400708-127.0.0.1:54347_-n_0000000000) [n:127.0.0.1:54347_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 189651 INFO  (zkCallback-102-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 190625 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 190625 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 190626 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 190626 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 190627 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 190627 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_74AB2677F5DB2089-001/control-001/cores/collection1'
   [junit4]   2> 190627 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 190628 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 190631 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 190640 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 190655 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 190687 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 190697 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 190930 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 190932 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 190933 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 190937 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 190980 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 190980 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 190980 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 190989 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:47484/solr_hdfs_home
   [junit4]   2> 190989 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 190990 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_74AB2677F5DB2089-001/control-001/cores/collection1], dataDir=[hdfs://localhost:47484/solr_hdfs_home/control_collection/core_node1/data/]
   [junit4]   2> 190990 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@36d505eb
   [junit4]   2> 190990 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:47484/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 191041 WARN  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
   [junit4]   2> 196043 WARN  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
   [junit4]   2> 201061 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:47484/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 201067 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:47484/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 201091 WARN  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:47484/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 201091 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:47484/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 201107 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 201107 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 201107 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new global HDFS BlockCache
   [junit4]   2> 201154 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 201154 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:47484/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 201155 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=6, maxMergeAtOnceExplicit=3, maxMergedSegmentMB=67.2705078125, floorSegmentMB=1.859375, forceMergeDeletesPctAllowed=0.8471949804612, segmentsPerTier=23.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7725834348124405
   [junit4]   2> 201198 INFO  (IPC Server handler 4 on 47484) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:35698 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-8ed71e16-9545-47f1-943f-b509f583feeb:NORMAL:127.0.0.1:54897|RBW], ReplicaUnderConstruction[[DISK]DS-9c6bef0f-f1a9-45cb-92b5-815dbd5f4a03:NORMAL:127.0.0.1:35698|FINALIZED]]} size 0
   [junit4]   2> 201203 INFO  (IPC Server handler 6 on 47484) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:54897 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9c6bef0f-f1a9-45cb-92b5-815dbd5f4a03:NORMAL:127.0.0.1:35698|FINALIZED], ReplicaUnderConstruction[[DISK]DS-2953dc3f-3138-45d5-be78-06abf999a567:NORMAL:127.0.0.1:54897|FINALIZED]]} size 0
   [junit4]   2> 201219 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@hdfs://localhost:47484/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4cec170); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 201219 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 201223 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:47484/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 201229 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 201240 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 201240 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 201240 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 201240 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 201241 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 201241 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 201242 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 201242 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 201242 WARN  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 201247 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 201248 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 201250 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 201252 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 201263 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: standard,/config,/admin/file,/admin/threads,/admin/system,/dump,/admin/logging,/update/json,/admin/luke,/admin/ping,/replication,/update/json/docs,/update,/admin/properties,/get,/schema,/admin/plugins,/update/csv,/admin/segments,/admin/mbeans
   [junit4]   2> 201265 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 201268 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 201268 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 201290 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms; 
   [junit4]   2> 201290 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 201293 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=10, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=0.3916015625, noCFSRatio=0.5928147530256638]
   [junit4]   2> 201407 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@hdfs://localhost:47484/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4cec170); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 201408 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 201408 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@a70efdb[collection1] main
   [junit4]   2> 201408 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 201416 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 201416 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 201420 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 201420 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 201420 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 201421 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 201421 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 201421 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 201424 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 201425 INFO  (searcherExecutor-622-thread-1-processing-s:shard1 r:core_node1 c:control_collection x:collection1 n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@a70efdb[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 201428 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 201428 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 201428 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 201428 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1522776801921728512
   [junit4]   2> 201428 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 1522776801921728512
   [junit4]   2> 201430 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 201432 INFO  (coreLoadExecutor-621-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 201432 INFO  (coreZkRegister-615-thread-1-processing-s:shard1 r:core_node1 c:control_collection x:collection1 n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:54347 collection:control_collection shard:shard1
   [junit4]   2> 201449 INFO  (coreZkRegister-615-thread-1-processing-s:shard1 r:core_node1 c:control_collection x:collection1 n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 201458 INFO  (coreZkRegister-615-thread-1-processing-s:shard1 r:core_node1 c:control_collection x:collection1 n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95173549299400708-core_node1-n_0000000000
   [junit4]   2> 201459 INFO  (coreZkRegister-615-thread-1-processing-s:shard1 r:core_node1 c:control_collection x:collection1 n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 201463 INFO  (coreZkRegister-615-thread-1-processing-s:shard1 r:core_node1 c:control_collection x:collection1 n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 201463 INFO  (coreZkRegister-615-thread-1-processing-s:shard1 r:core_node1 c:control_collection x:collection1 n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 201463 INFO  (OverseerStateUpdate-95173549299400708-127.0.0.1:54347_-n_0000000000) [n:127.0.0.1:54347_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 201465 INFO  (coreZkRegister-615-thread-1-processing-s:shard1 r:core_node1 c:control_collection x:collection1 n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:54347/collection1/
   [junit4]   2> 201465 INFO  (coreZkRegister-615-thread-1-processing-s:shard1 r:core_node1 c:control_collection x:collection1 n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 201465 INFO  (coreZkRegister-615-thread-1-processing-s:shard1 r:core_node1 c:control_collection x:collection1 n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:54347/collection1/ has no replicas
   [junit4]   2> 201469 INFO  (zkCallback-102-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 201469 INFO  (coreZkRegister-615-thread-1-processing-s:shard1 r:core_node1 c:control_collection x:collection1 n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 201481 INFO  (coreZkRegister-615-thread-1-processing-s:shard1 r:core_node1 c:control_collection x:collection1 n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 201516 INFO  (coreZkRegister-615-thread-1-processing-s:shard1 r:core_node1 c:control_collection x:collection1 n:127.0.0.1:54347_) [n:127.0.0.1:54347_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:54347/collection1/ shard1
   [junit4]   2> 201517 INFO  (OverseerStateUpdate-95173549299400708-127.0.0.1:54347_-n_0000000000) [n:127.0.0.1:54347_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:54347",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 201622 INFO  (zkCallback-102-thread-1-processing-n:127.0.0.1:54347_) [n:127.0.0.1:54347_    ] o.a.s.c.c.ZkStateReader A clust

[...truncated too long message...]

zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:328)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:325)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:325)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:108)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:59)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:389)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:264)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:232)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 3591356 WARN  (zkCallback-2734-thread-1-processing-n:127.0.0.1:38987_solr) [n:127.0.0.1:38987_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 3591356 WARN  (zkCallback-2738-thread-1-processing-n:127.0.0.1:57745_solr) [n:127.0.0.1:57745_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 3591356 WARN  (zkCallback-2742-thread-1-processing-n:127.0.0.1:52696_solr) [n:127.0.0.1:52696_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 3591357 WARN  (zkCallback-2716-thread-1-processing-n:127.0.0.1:51889_solr) [n:127.0.0.1:51889_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 3591357 WARN  (zkCallback-2730-thread-1-processing-n:127.0.0.1:48074_solr) [n:127.0.0.1:48074_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 3591359 INFO  (jetty-launcher-2698-thread-6) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6177e8d{/solr,null,UNAVAILABLE}
   [junit4]   2> 3591363 INFO  (jetty-launcher-2698-thread-10) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@b5017d4{/solr,null,UNAVAILABLE}
   [junit4]   2> 3591365 INFO  (jetty-launcher-2698-thread-7) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2e4cde76{/solr,null,UNAVAILABLE}
   [junit4]   2> 3591365 INFO  (jetty-launcher-2698-thread-8) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@30c91ceb{/solr,null,UNAVAILABLE}
   [junit4]   2> 3591365 INFO  (jetty-launcher-2698-thread-9) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6207295e{/solr,null,UNAVAILABLE}
   [junit4]   2> 3591368 INFO  (TEST-TestAuthenticationFramework.testStopAllStartAll-seed#[74AB2677F5DB2089]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:56696 56696
   [junit4]   2> 3591577 INFO  (Thread-10914) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:56696 56696
   [junit4]   2> 3591578 WARN  (Thread-10914) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	11	/solr/aliases.json
   [junit4]   2> 	9	/solr/security.json
   [junit4]   2> 	8	/solr/configs/solrCloudCollectionConfig
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	11	/solr/clusterstate.json
   [junit4]   2> 	8	/solr/collections/testStopAllStartAllCollection/state.json
   [junit4]   2> 	3	/solr/overseer_elect/election/95173768083603468-127.0.0.1:51889_solr-n_0000000001
   [junit4]   2> 	2	/solr/overseer_elect/election/95173768083603467-127.0.0.1:48074_solr-n_0000000002
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	11	/solr/collections
   [junit4]   2> 	7	/solr/live_nodes
   [junit4]   2> 	2	/solr/overseer/queue
   [junit4]   2> 	2	/solr/overseer/collection-queue-work
   [junit4]   2> 
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_74AB2677F5DB2089-001
   [junit4]   2> Jan 08, 2016 7:01:55 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {timestamp=PostingsFormat(name=LuceneVarGapFixedInterval), range_facet_l_dv=FSTOrd50, id=FSTOrd50, intDefault=PostingsFormat(name=LuceneFixedGap), range_facet_i_dv=PostingsFormat(name=LuceneVarGapFixedInterval), range_facet_l=PostingsFormat(name=LuceneVarGapFixedInterval), _version_=PostingsFormat(name=LuceneFixedGap), multiDefault=PostingsFormat(name=LuceneVarGapFixedInterval)}, docValues:{timestamp=DocValuesFormat(name=Direct), range_facet_l_dv=DocValuesFormat(name=Lucene54), range_facet_i_dv=DocValuesFormat(name=Direct)}, sim=DefaultSimilarity, locale=ar_AE, timezone=Canada/Saskatchewan
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=245502536,total=524812288
   [junit4]   2> NOTE: All tests run in this JVM: [TestMaxScoreQueryParser, SearchHandlerTest, TestCopyFieldCollectionResource, TestUtils, LeaderFailoverAfterPartitionTest, HardAutoCommitTest, TestDistributedMissingSort, ConjunctionSolrSpellCheckerTest, UnloadDistributedZkTest, TestExpandComponent, TestInitParams, HdfsRecoveryZkTest, BadIndexSchemaTest, AnalysisErrorHandlingTest, TestSort, TestRuleBasedAuthorizationPlugin, TestRecovery, SolrTestCaseJ4Test, DistributedFacetPivotLongTailTest, SuggesterTest, TestPHPSerializedResponseWriter, TestCloudManagedSchemaConcurrent, LukeRequestHandlerTest, RemoteQueryErrorTest, TestCharFilters, BadComponentTest, BitVectorTest, SliceStateTest, TestSolrXml, TestElisionMultitermQuery, CursorMarkTest, NoCacheHeaderTest, ShowFileRequestHandlerTest, RequestHandlersTest, CoreAdminHandlerTest, ActionThrottleTest, HdfsDirectoryFactoryTest, TestConfigSetImmutable, TestReRankQParserPlugin, DocExpirationUpdateProcessorFactoryTest, RecoveryZkTest, HdfsCollectionsAPIDistributedZkTest, TriLevelCompositeIdRoutingTest, TestDefaultStatsCache, RollingRestartTest, TestDynamicFieldResource, TestTestInjection, FieldAnalysisRequestHandlerTest, ClusterStateTest, CachingDirectoryFactoryTest, TestSolrJ, TestJmxIntegration, HdfsDirectoryTest, FullSolrCloudDistribCmdsTest, StressHdfsTest, BasicDistributedZk2Test, TestCollationField, TestTolerantSearch, TestFieldSortValues, TestLeaderInitiatedRecoveryThread, CoreAdminCreateDiscoverTest, TestRestoreCore, TestFastWriter, TestOverriddenPrefixQueryForCustomFieldType, TestRestManager, TestSuggestSpellingConverter, OverseerStatusTest, TestIndexSearcher, ScriptEngineTest, TestGroupingSearch, TestUseDocValuesAsStored2, XmlUpdateRequestHandlerTest, AsyncCallRequestStatusResponseTest, TestFuzzyAnalyzedSuggestions, ShardSplitTest, TestRebalanceLeaders, TestConfigSetsAPIZkFailure, TestJmxMonitoredMap, TestRawResponseWriter, PreAnalyzedUpdateProcessorTest, DocValuesMultiTest, UpdateParamsTest, TestXIncludeConfig, TestCollectionAPI, PKIAuthenticationIntegrationTest, TestDynamicLoading, BlockJoinFacetDistribTest, TestManagedSchemaFieldResource, TestNonDefinedSimilarityFactory, TestReversedWildcardFilterFactory, MigrateRouteKeyTest, TestAuthenticationFramework]
   [junit4] Completed [449/563 (6!)] on J2 in 131.19s, 7 tests, 1 failure <<< FAILURES!

[...truncated 384 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1444: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1000: There were test failures: 563 suites (7 ignored), 2236 tests, 1 error, 5 failures, 33 ignored (18 assumptions) [seed: 74AB2677F5DB2089]

Total time: 250 minutes 51 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1068 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1068/

All tests passed

Build Log:
[...truncated 746 lines...]
   [junit4] JVM J1: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/temp/junit4-J1-20160107_171355_250.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] WARN: Unhandled exception in event serialization. -> java.security.PrivilegedActionException: java.io.IOException: No space left on device
   [junit4] 	at java.security.AccessController.doPrivileged(Native Method)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.events.Serializer.flushQueue(Serializer.java:96)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.events.Serializer.serialize(Serializer.java:81)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.slave.RunListenerEmitter.testFailure(RunListenerEmitter.java:54)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.slave.NoExceptionRunListenerDecorator.testFailure(NoExceptionRunListenerDecorator.java:55)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.slave.BeforeAfterRunListenerDecorator.testFailure(BeforeAfterRunListenerDecorator.java:60)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.slave.OrderedRunNotifier$4.notifyListener(OrderedRunNotifier.java:129)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.slave.OrderedRunNotifier$SafeNotifier.run(OrderedRunNotifier.java:63)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.slave.OrderedRunNotifier.fireTestFailure(OrderedRunNotifier.java:126)
   [junit4] 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$SubNotifier.fireTestFailure(ThreadLeakControl.java:183)
   [junit4] 	at com.carrotsearch.randomizedtesting.RandomizedRunner.fireTestFailure(RandomizedRunner.java:798)
   [junit4] 	at com.carrotsearch.randomizedtesting.RandomizedRunner.fireTestFailure(RandomizedRunner.java:795)
   [junit4] 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:886)
   [junit4] 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
   [junit4] 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
   [junit4] 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
   [junit4] 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4] 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4] 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4] 	at com.carr
   [junit4] otsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4] 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4] 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4] 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4] 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4] 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4] 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4] 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4] 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4] 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4] 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)
   [junit4] Caused by: java.io.IOException: No space left on device
   [junit4] 	at java.io.FileOutputStream.writeBytes(Native Method)
   [junit4] 	at java.io.FileOutputStream.write(FileOutputStream.java:345)
   [junit4] 	at java.io.BufferedOutputStream.write(BufferedOutputStream.java:122)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.slave.EventsOutputStream.write(EventsOutputStream.java:28)
   [junit4] 	at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
   [junit4] 	at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:282)
   [junit4] 	at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
   [junit4] 	at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:135)
   [junit4] 	at java.io.OutputStreamWriter.write(OutputStreamWriter.java:220)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.gson.stream.JsonWriter.string(JsonWriter.java:565)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.gson.stream.JsonWriter.value(JsonWriter.java:414)
   [junit4] 	at com.carrotsearch.ant.task
   [junit4] s.junit4.events.FailureEvent.serialize(FailureEvent.java:46)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.events.Serializer$2.run(Serializer.java:101)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.events.Serializer$2.run(Serializer.java:96)
   [junit4] 	... 31 more
   [junit4] <<< JVM J1: EOF ----

[...truncated 6 lines...]
   [junit4] ERROR: JVM J0 ended with an exception, command line: /x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/jre/bin/java -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps -ea -esa -Dtests.prefix=tests -Dtests.seed=5C03CBED3238C0EF -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.luceneMatchVersion=5.5.0 -Dtests.cleanthreads=perMethod -Djava.util.logging.config.file=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/tools/junit4/logging.properties -Dtests.nightly=true -Dtests.weekly=false -Dtests.monster=false -Dtests.slow=true -Dtests.asserts=true -Dtests.multiplier=2 -DtempDir=./temp -Djava.io.tmpdir=./temp -Djunit4.tempDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/temp -Dcommon.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene -Dclover.db.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/clover/db -Djava.security.policy=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/tools/junit4/tests.policy -Dtests.LUCENE_VERSION=5.5.0 -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Djdk.map.althashing.threshold=0 -Djunit4.childvm.cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J0 -Djunit4.childvm.id=0 -Djunit4.childvm.count=3 -Dtests.leaveTemporary=false -Dtests.filterstacks=true -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -Dfile.encoding=ISO-8859-1 -classpath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/codecs/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/test-framework/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/test-framework/lib/junit-4.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/test-framework/lib/randomizedtesting-runner-2.3.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/test:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.2.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/temp/junit4-J0-20160107_171355_250.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/temp/junit4-J0-20160107_171355_250.suites -stdin
   [junit4] ERROR: JVM J0 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)
   [junit4] ERROR: JVM J1 ended with an exception, command line: /x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/jre/bin/java -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps -ea -esa -Dtests.prefix=tests -Dtests.seed=5C03CBED3238C0EF -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.luceneMatchVersion=5.5.0 -Dtests.cleanthreads=perMethod -Djava.util.logging.config.file=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/tools/junit4/logging.properties -Dtests.nightly=true -Dtests.weekly=false -Dtests.monster=false -Dtests.slow=true -Dtests.asserts=true -Dtests.multiplier=2 -DtempDir=./temp -Djava.io.tmpdir=./temp -Djunit4.tempDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/temp -Dcommon.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene -Dclover.db.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/clover/db -Djava.security.policy=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/tools/junit4/tests.policy -Dtests.LUCENE_VERSION=5.5.0 -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Djdk.map.althashing.threshold=0 -Djunit4.childvm.cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1 -Djunit4.childvm.id=1 -Djunit4.childvm.count=3 -Dtests.leaveTemporary=false -Dtests.filterstacks=true -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -Dfile.encoding=ISO-8859-1 -classpath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/codecs/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/test-framework/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/test-framework/lib/junit-4.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/test-framework/lib/randomizedtesting-runner-2.3.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/test:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.2.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/temp/junit4-J1-20160107_171355_250.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/temp/junit4-J1-20160107_171355_250.suites -stdin
   [junit4] ERROR: JVM J1 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)
   [junit4] ERROR: JVM J2 ended with an exception, command line: /x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/jre/bin/java -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps -ea -esa -Dtests.prefix=tests -Dtests.seed=5C03CBED3238C0EF -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.luceneMatchVersion=5.5.0 -Dtests.cleanthreads=perMethod -Djava.util.logging.config.file=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/tools/junit4/logging.properties -Dtests.nightly=true -Dtests.weekly=false -Dtests.monster=false -Dtests.slow=true -Dtests.asserts=true -Dtests.multiplier=2 -DtempDir=./temp -Djava.io.tmpdir=./temp -Djunit4.tempDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/temp -Dcommon.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene -Dclover.db.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/clover/db -Djava.security.policy=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/tools/junit4/tests.policy -Dtests.LUCENE_VERSION=5.5.0 -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Djdk.map.althashing.threshold=0 -Djunit4.childvm.cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J2 -Djunit4.childvm.id=2 -Djunit4.childvm.count=3 -Dtests.leaveTemporary=false -Dtests.filterstacks=true -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -Dfile.encoding=ISO-8859-1 -classpath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/codecs/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/test-framework/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/test-framework/lib/junit-4.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/test-framework/lib/randomizedtesting-runner-2.3.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/test:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.2.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/temp/junit4-J2-20160107_171355_250.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/temp/junit4-J2-20160107_171355_250.suites -stdin
   [junit4] ERROR: JVM J2 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:50: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1444: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1000: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 22 minutes 43 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1067 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1067/

All tests passed

Build Log:
[...truncated 10091 lines...]
    [javac] Compiling 613 source files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/classes/test
    [javac] /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test/org/apache/solr/cloud/TestMiniSolrCloudCluster.java:103: error: cannot find symbol
    [javac]     collectionProperties.putIfAbsent(CoreDescriptor.CORE_CONFIG, "solrconfig-tlog.xml");
    [javac]                         ^
    [javac]   symbol:   method putIfAbsent(String,String)
    [javac]   location: variable collectionProperties of type Map<String,String>
    [javac] /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test/org/apache/solr/cloud/TestMiniSolrCloudCluster.java:104: error: cannot find symbol
    [javac]     collectionProperties.putIfAbsent("solr.tests.maxBufferedDocs", "100000");
    [javac]                         ^
    [javac]   symbol:   method putIfAbsent(String,String)
    [javac]   location: variable collectionProperties of type Map<String,String>
    [javac] /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test/org/apache/solr/cloud/TestMiniSolrCloudCluster.java:105: error: cannot find symbol
    [javac]     collectionProperties.putIfAbsent("solr.tests.ramBufferSizeMB", "100");
    [javac]                         ^
    [javac]   symbol:   method putIfAbsent(String,String)
    [javac]   location: variable collectionProperties of type Map<String,String>
    [javac] /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test/org/apache/solr/cloud/TestMiniSolrCloudCluster.java:107: error: cannot find symbol
    [javac]     collectionProperties.putIfAbsent("solr.tests.mergePolicy", "org.apache.lucene.index.TieredMergePolicy");
    [javac]                         ^
    [javac]   symbol:   method putIfAbsent(String,String)
    [javac]   location: variable collectionProperties of type Map<String,String>
    [javac] /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test/org/apache/solr/cloud/TestMiniSolrCloudCluster.java:108: error: cannot find symbol
    [javac]     collectionProperties.putIfAbsent("solr.tests.mergeScheduler", "org.apache.lucene.index.ConcurrentMergeScheduler");
    [javac]                         ^
    [javac]   symbol:   method putIfAbsent(String,String)
    [javac]   location: variable collectionProperties of type Map<String,String>
    [javac] /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test/org/apache/solr/cloud/TestMiniSolrCloudCluster.java:109: error: cannot find symbol
    [javac]     collectionProperties.putIfAbsent("solr.directoryFactory", (persistIndex ? "solr.StandardDirectoryFactory" : "solr.RAMDirectoryFactory"));
    [javac]                         ^
    [javac]   symbol:   method putIfAbsent(String,String)
    [javac]   location: variable collectionProperties of type Map<String,String>
    [javac] Note: Some input files use or override a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
    [javac] Note: Some input files use unchecked or unsafe operations.
    [javac] Note: Recompile with -Xlint:unchecked for details.
    [javac] 6 errors

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:808: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:822: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1956: Compile failed; see the compiler error output for details.

Total time: 133 minutes 9 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1066 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1066/

All tests passed

Build Log:
[...truncated 11006 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20160105_082407_512.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid4782.hprof ...
   [junit4] Heap dump file created [688171648 bytes in 8.643 secs]
   [junit4] <<< JVM J1: EOF ----

   [junit4] JVM J1: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20160105_082407_512.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] WARN: Unhandled exception in event serialization. -> java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] <<< JVM J1: EOF ----

[...truncated 915 lines...]
   [junit4] ERROR: JVM J1 ended with an exception, command line: /x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/jre/bin/java -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps -XX:MaxPermSize=192m -ea -esa -Dtests.prefix=tests -Dtests.seed=89100D3D531CBEFD -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.luceneMatchVersion=5.5.0 -Dtests.cleanthreads=perClass -Djava.util.logging.config.file=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/tools/junit4/logging.properties -Dtests.nightly=true -Dtests.weekly=false -Dtests.monster=false -Dtests.slow=true -Dtests.asserts=true -Dtests.multiplier=2 -DtempDir=./temp -Djava.io.tmpdir=./temp -Djunit4.tempDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp -Dcommon.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene -Dclover.db.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/clover/db -Djava.security.policy=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/tools/junit4/solr-tests.policy -Dtests.LUCENE_VERSION=5.5.0 -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Djdk.map.althashing.threshold=0 -Djunit4.childvm.cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1 -Djunit4.childvm.id=1 -Djunit4.childvm.count=3 -Dtests.leaveTemporary=false -Dtests.filterstacks=true -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -Dfile.encoding=UTF-8 -classpath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-test-framework/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/test-framework/lib/junit4-ant-2.3.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/test-framework/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/codecs/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-solrj/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/common/lucene-analyzers-common-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/codecs/lucene-codecs-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/backward-codecs/lucene-backward-codecs-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/highlighter/lucene-highlighter-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/memory/lucene-memory-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/misc/lucene-misc-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/spatial/lucene-spatial-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/expressions/lucene-expressions-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/suggest/lucene-suggest-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/grouping/lucene-grouping-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/queries/lucene-queries-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/queryparser/lucene-queryparser-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/join/lucene-join-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/sandbox/lucene-sandbox-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/antlr4-runtime-4.5.1-1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/asm-5.0.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/asm-commons-5.0.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-cli-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-codec-1.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-configuration-1.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-exec-1.3.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-fileupload-1.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-lang-2.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/concurrentlinkedhashmap-lru-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/dom4j-1.6.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/guava-14.0.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/hadoop-annotations-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/hadoop-auth-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/hadoop-common-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/hadoop-hdfs-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/hppc-0.7.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/htrace-core-3.0.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/jackson-core-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/jackson-dataformat-smile-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/joda-time-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/log4j-1.2.17.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/org.restlet-2.3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/org.restlet.ext.servlet-2.3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/protobuf-java-2.5.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/slf4j-log4j12-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/spatial4j-0.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/t-digest-3.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/commons-io-2.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/httpclient-4.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/httpcore-4.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/httpmime-4.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/noggit-0.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/slf4j-api-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/stax2-api-3.1.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/woodstox-core-asl-4.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/zookeeper-3.4.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/javax.servlet-api-3.1.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-continuation-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-deploy-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-http-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-io-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-jmx-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-rewrite-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-security-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-server-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-servlet-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-servlets-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-util-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-webapp-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-xml-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/example/example-DIH/solr/db/lib/derby-10.9.1.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/test-framework/lib/junit-4.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/test-framework/lib/randomizedtesting-runner-2.3.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/antlr-2.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/antlr4-runtime-4.5.1-1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-core-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-core-api-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-core-avl-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-core-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-i18n-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptor-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-admin-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.2.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20160105_082407_512.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20160105_082407_512.suites -stdin
   [junit4] ERROR: JVM J1 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1444: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1000: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 222 minutes 2 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
ERROR: Failed to archive {solr/build/solr-core/test/temp/junit4-J0-20160105_082407_509.events=solr/build/solr-core/test/temp/junit4-J0-20160105_082407_509.events, README.txt=README.txt, solr/build/solr-core/test/temp/junit4-J1-20160105_082407_512.events=solr/build/solr-core/test/temp/junit4-J1-20160105_082407_512.events, heapdumps/java_pid4782.hprof=heapdumps/java_pid4782.hprof, solr/build/solr-core/test/temp/junit4-J2-20160105_082407_512.events=solr/build/solr-core/test/temp/junit4-J2-20160105_082407_512.events} due to internal error; falling back to full archiving
java.lang.NullPointerException
	at jsync.protocol.ChangeInputStream.read(ChangeInputStream.java:69)
	at com.cloudbees.jenkins.plugins.jsync.archiver.MD5DigestingInputStream.read(MD5DigestingInputStream.java:39)
	at com.google.common.io.LimitInputStream.read(LimitInputStream.java:79)
	at java.io.FilterInputStream.read(FilterInputStream.java:107)
	at com.google.common.io.ByteStreams.copy(ByteStreams.java:193)
	at jsync.protocol.FileSequenceReader.read(FileSequenceReader.java:35)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.remoteSync(JSyncArtifactManager.java:148)
	at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.archive(JSyncArtifactManager.java:72)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:219)
	at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:74)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:776)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:723)
	at hudson.model.Build$BuildExecution.post2(Build.java:183)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:670)
	at hudson.model.Run.execute(Run.java:1763)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:98)
	at hudson.model.Executor.run(Executor.java:381)
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1065 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1065/

1 tests failed.
FAILED:  org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=18754, name=Thread-13368, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=18754, name=Thread-13368, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:55827/collection1
	at __randomizedtesting.SeedInfo.seed([52BE79E325438F54]:0)
	at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest$1IndexThread.run(FullSolrCloudDistribCmdsTest.java:645)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:55827/collection1
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:167)
	at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest$1IndexThread.run(FullSolrCloudDistribCmdsTest.java:643)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
	... 5 more




Build Log:
[...truncated 10682 lines...]
   [junit4] Suite: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/init-core-data-001
   [junit4]   2> 1607281 INFO  (SUITE-FullSolrCloudDistribCmdsTest-seed#[52BE79E325438F54]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 1607287 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1607288 INFO  (Thread-13198) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1607288 INFO  (Thread-13198) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1607388 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.ZkTestServer start zk server on port:40033
   [junit4]   2> 1607388 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1607388 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1607391 INFO  (zkCallback-1002-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6688fe6a name:ZooKeeperConnection Watcher:127.0.0.1:40033 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1607391 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1607391 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1607391 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1607393 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1607394 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1607400 INFO  (zkCallback-1003-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@e501359 name:ZooKeeperConnection Watcher:127.0.0.1:40033/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1607400 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1607400 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1607400 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1607402 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1607404 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1607405 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1607406 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1607406 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1607409 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 1607409 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1607411 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1607411 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1607412 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1607413 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1607414 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1607414 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1607416 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1607416 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1607418 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1607418 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1607420 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1607420 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1607421 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1607422 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1607423 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1607423 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1607425 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1607425 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1607712 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/control-001/cores/collection1
   [junit4]   2> 1607714 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1607716 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@963e712{/,null,AVAILABLE}
   [junit4]   2> 1607717 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.e.j.s.ServerConnector Started ServerConnector@43aed02d{HTTP/1.1}{127.0.0.1:55827}
   [junit4]   2> 1607717 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.e.j.s.Server Started @1610706ms
   [junit4]   2> 1607718 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=55827, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/tempDir-001/control/data, hostContext=/, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/control-001/cores}
   [junit4]   2> 1607718 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@41692a49
   [junit4]   2> 1607718 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/control-001'
   [junit4]   2> 1607719 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1607719 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1607719 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1607720 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1607721 INFO  (zkCallback-1004-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2ccfc372 name:ZooKeeperConnection Watcher:127.0.0.1:40033/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1607721 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1607722 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1607723 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1607723 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/control-001/solr.xml
   [junit4]   2> 1607737 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/control-001/cores
   [junit4]   2> 1607737 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.CoreContainer New CoreContainer 1996720548
   [junit4]   2> 1607737 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/control-001]
   [junit4]   2> 1607737 WARN  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/control-001/lib
   [junit4]   2> 1607738 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1607739 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1607739 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1607739 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1607739 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40033/solr
   [junit4]   2> 1607740 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1607740 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1607740 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1607742 INFO  (zkCallback-1006-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@27ec1198 name:ZooKeeperConnection Watcher:127.0.0.1:40033 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1607742 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1607743 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1607744 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:55827_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1607745 INFO  (zkCallback-1007-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@416c8ef7 name:ZooKeeperConnection Watcher:127.0.0.1:40033/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1607745 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:55827_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1607746 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:55827_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1607748 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:55827_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1607751 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:55827_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1607753 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:55827_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1607755 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:55827_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1607756 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:55827_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1607758 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:55827_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1607759 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:55827_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1607760 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:55827_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1607761 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:55827_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55827_
   [junit4]   2> 1607761 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:55827_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:55827_
   [junit4]   2> 1607763 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:55827_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1607764 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:55827_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1607765 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:55827_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1607766 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:55827_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95151506224381956-127.0.0.1:55827_-n_0000000000
   [junit4]   2> 1607767 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:55827_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:55827_
   [junit4]   2> 1607767 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:55827_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1607769 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:55827_    ] o.a.s.c.Overseer Overseer (id=95151506224381956-127.0.0.1:55827_-n_0000000000) starting
   [junit4]   2> 1607771 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:55827_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1607776 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:55827_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1607777 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:55827_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1607777 INFO  (OverseerCollectionConfigSetProcessor-95151506224381956-127.0.0.1:55827_-n_0000000000) [n:127.0.0.1:55827_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1607778 INFO  (OverseerStateUpdate-95151506224381956-127.0.0.1:55827_-n_0000000000) [n:127.0.0.1:55827_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1607792 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:55827_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1607792 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:55827_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1607793 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:55827_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/control-001/cores
   [junit4]   2> 1607794 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:55827_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {shard=, dataDir=data/, transient=false, loadOnStartup=true, config=solrconfig.xml, collection=control_collection, configSetProperties=configsetprops.json, coreNodeName=, schema=schema.xml, name=collection1}
   [junit4]   2> 1607794 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:55827_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/control-001/cores/collection1
   [junit4]   2> 1607794 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:55827_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1607795 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:55827_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1607795 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:55827_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1607795 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1607795 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1607797 INFO  (OverseerStateUpdate-95151506224381956-127.0.0.1:55827_-n_0000000000) [n:127.0.0.1:55827_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:55827_",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:55827",
   [junit4]   2>   "state":"down"} current state version: 0
   [junit4]   2> 1607797 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1607797 INFO  (OverseerStateUpdate-95151506224381956-127.0.0.1:55827_-n_0000000000) [n:127.0.0.1:55827_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:55827_",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:55827",
   [junit4]   2>   "state":"down"}
   [junit4]   2> 1607798 INFO  (OverseerStateUpdate-95151506224381956-127.0.0.1:55827_-n_0000000000) [n:127.0.0.1:55827_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1607798 INFO  (OverseerStateUpdate-95151506224381956-127.0.0.1:55827_-n_0000000000) [n:127.0.0.1:55827_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1607800 INFO  (zkCallback-1007-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1608798 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1608798 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1608798 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1608799 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 1608799 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1608800 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/control-001/cores/collection1'
   [junit4]   2> 1608800 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1608800 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1608802 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1608809 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1608824 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1608857 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1608865 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1609104 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1609107 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1609108 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1609117 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1609137 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1609137 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1609137 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1609138 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/control-001/cores/collection1/data/]
   [junit4]   2> 1609138 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@339b9
   [junit4]   2> 1609138 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/control-001/cores/collection1/data
   [junit4]   2> 1609139 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1609139 WARN  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1609139 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/control-001/cores/collection1/data/index
   [junit4]   2> 1609139 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@832a0d5
   [junit4]   2> 1609140 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@33dd0427 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@733bb70a),segFN=segments_1,generation=1}
   [junit4]   2> 1609140 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1609140 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/control-001/cores/collection1/data/
   [junit4]   2> 1609140 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1609145 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1609146 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1609146 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1609146 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1609146 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1609146 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1609147 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1609147 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1609147 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1609147 WARN  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 1609150 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1609152 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1609153 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1609155 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1609159 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/plugins,/admin/ping,/schema,/admin/logging,/config,/admin/threads,/update/json,/dump,/update,/admin/segments,/admin/file,/admin/properties,/admin/luke,/admin/mbeans,/update/json/docs,/update/csv,/admin/system,/get,standard,/replication
   [junit4]   2> 1609160 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1609161 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1609161 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1609162 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1609162 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1609163 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=3, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=0.9013671875, noCFSRatio=0.6860541864641313]
   [junit4]   2> 1609163 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@33dd0427 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@733bb70a),segFN=segments_1,generation=1}
   [junit4]   2> 1609163 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1609163 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@7f2bc9cd[collection1] main
   [junit4]   2> 1609163 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 1609164 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1609164 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1609165 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1609165 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1609165 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1609165 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1609165 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1609165 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1609166 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1609166 INFO  (searcherExecutor-3415-thread-1-processing-x:collection1 n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7f2bc9cd[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1609166 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1609167 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1609167 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1609167 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1522424101557239808
   [junit4]   2> 1609169 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1522424101557239808
   [junit4]   2> 1609169 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1609170 INFO  (coreLoadExecutor-3414-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1609170 INFO  (coreZkRegister-3408-thread-1-processing-x:collection1 n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:55827 collection:control_collection shard:shard1
   [junit4]   2> 1609171 INFO  (coreZkRegister-3408-thread-1-processing-x:collection1 n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1609175 INFO  (coreZkRegister-3408-thread-1-processing-x:collection1 n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95151506224381956-core_node1-n_0000000000
   [junit4]   2> 1609176 INFO  (coreZkRegister-3408-thread-1-processing-x:collection1 n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 1609178 INFO  (coreZkRegister-3408-thread-1-processing-x:collection1 n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1609178 INFO  (coreZkRegister-3408-thread-1-processing-x:collection1 n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1609178 INFO  (coreZkRegister-3408-thread-1-processing-x:collection1 n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:55827/collection1/
   [junit4]   2> 1609178 INFO  (coreZkRegister-3408-thread-1-processing-x:collection1 n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1609178 INFO  (OverseerStateUpdate-95151506224381956-127.0.0.1:55827_-n_0000000000) [n:127.0.0.1:55827_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1609178 INFO  (coreZkRegister-3408-thread-1-processing-x:collection1 n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:55827/collection1/ has no replicas
   [junit4]   2> 1609179 INFO  (coreZkRegister-3408-thread-1-processing-x:collection1 n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1609182 INFO  (coreZkRegister-3408-thread-1-processing-x:collection1 n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1609184 INFO  (coreZkRegister-3408-thread-1-processing-x:collection1 n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:55827/collection1/ shard1
   [junit4]   2> 1609184 INFO  (OverseerStateUpdate-95151506224381956-127.0.0.1:55827_-n_0000000000) [n:127.0.0.1:55827_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:55827",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1609287 INFO  (zkCallback-1007-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1609297 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1609297 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1609299 INFO  (zkCallback-1009-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5d70a2ff name:ZooKeeperConnection Watcher:127.0.0.1:40033/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1609299 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1609300 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1609300 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1609302 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1609302 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 1609302 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1609302 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1609303 INFO  (zkCallback-1010-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1a37bdfb name:ZooKeeperConnection Watcher:127.0.0.1:40033/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1609304 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1609304 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1609306 INFO  (OverseerStateUpdate-95151506224381956-127.0.0.1:55827_-n_0000000000) [n:127.0.0.1:55827_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"3",
   [junit4]   2>   "stateFormat":"2"} current state version: 2
   [junit4]   2> 1609307 INFO  (OverseerStateUpdate-95151506224381956-127.0.0.1:55827_-n_0000000000) [n:127.0.0.1:55827_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1609335 INFO  (coreZkRegister-3408-thread-1-processing-x:collection1 n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:55827/collection1/ and leader is http://127.0.0.1:55827/collection1/
   [junit4]   2> 1609335 INFO  (coreZkRegister-3408-thread-1-processing-x:collection1 n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:55827
   [junit4]   2> 1609335 INFO  (coreZkRegister-3408-thread-1-processing-x:collection1 n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1609335 INFO  (coreZkRegister-3408-thread-1-processing-x:collection1 n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1609335 INFO  (coreZkRegister-3408-thread-1-processing-x:collection1 n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:55827_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1609337 INFO  (OverseerStateUpdate-95151506224381956-127.0.0.1:55827_-n_0000000000) [n:127.0.0.1:55827_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:55827_",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"3",
   [junit4]   2>   "base_url":"http://127.0.0.1:55827",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 1609338 INFO  (OverseerStateUpdate-95151506224381956-127.0.0.1:55827_-n_0000000000) [n:127.0.0.1:55827_    ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:55827_",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"3",
   [junit4]   2>   "base_url":"http://127.0.0.1:55827",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1609339 INFO  (OverseerStateUpdate-95151506224381956-127.0.0.1:55827_-n_0000000000) [n:127.0.0.1:55827_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 1609443 INFO  (zkCallback-1009-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1609443 INFO  (zkCallback-1007-thread-1-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1609647 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/shard-1-001/cores/collection1
   [junit4]   2> 1609648 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/shard-1-001
   [junit4]   2> 1609649 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1609652 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@11b735d7{/,null,AVAILABLE}
   [junit4]   2> 1609652 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.e.j.s.ServerConnector Started ServerConnector@23baf298{HTTP/1.1}{127.0.0.1:58063}
   [junit4]   2> 1609652 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.e.j.s.Server Started @1612641ms
   [junit4]   2> 1609653 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/shard-1-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/tempDir-001/jetty1, hostContext=/, hostPort=58063, solrconfig=solrconfig.xml}
   [junit4]   2> 1609653 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@41692a49
   [junit4]   2> 1609653 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/shard-1-001'
   [junit4]   2> 1609653 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1609653 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1609654 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1609654 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1609656 INFO  (zkCallback-1011-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5e342707 name:ZooKeeperConnection Watcher:127.0.0.1:40033/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1609656 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1609656 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1609657 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1609658 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/shard-1-001/solr.xml
   [junit4]   2> 1609673 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/shard-1-001/cores
   [junit4]   2> 1609674 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.CoreContainer New CoreContainer 1570975802
   [junit4]   2> 1609674 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/shard-1-001]
   [junit4]   2> 1609674 WARN  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/shard-1-001/lib
   [junit4]   2> 1609674 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1609676 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1609676 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1609676 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1609676 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40033/solr
   [junit4]   2> 1609677 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1609677 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1609677 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1609680 INFO  (zkCallback-1013-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5aa6120 name:ZooKeeperConnection Watcher:127.0.0.1:40033 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1609680 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1609681 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1609683 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:58063_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1609684 INFO  (zkCallback-1014-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@56bf1408 name:ZooKeeperConnection Watcher:127.0.0.1:40033/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1609685 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:58063_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1609691 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:58063_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1610700 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:58063_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58063_
   [junit4]   2> 1610701 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:58063_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:58063_
   [junit4]   2> 1610704 INFO  (zkCallback-1007-thread-2-processing-n:127.0.0.1:55827_) [n:127.0.0.1:55827_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1610704 INFO  (zkCallback-1014-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1610704 INFO  (zkCallback-1009-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1610706 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:58063_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1610706 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:58063_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95151506224381961-127.0.0.1:58063_-n_0000000001
   [junit4]   2> 1610707 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:58063_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/95151506224381956-127.0.0.1:55827_-n_0000000000 to know if I could be the leader
   [junit4]   2> 1610725 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:58063_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1610725 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:58063_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1610726 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:58063_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/shard-1-001/cores
   [junit4]   2> 1610727 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:58063_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {loadOnStartup=true, configSetProperties=configsetprops.json, shard=, name=collection1, dataDir=data/, transient=false, coreNodeName=, collection=collection1, config=solrconfig.xml, schema=schema.xml}
   [junit4]   2> 1610727 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:58063_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/shard-1-001/cores/collection1
   [junit4]   2> 1610727 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:58063_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1610729 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:58063_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1610729 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[52BE79E325438F54]) [n:127.0.0.1:58063_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1610729 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1610729 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1610731 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 1610731 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch [collection1]
   [junit4]   2> 1610732 INFO  (OverseerStateUpdate-95151506224381956-127.0.0.1:55827_-n_0000000000) [n:127.0.0.1:55827_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:58063_",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"3",
   [junit4]   2>   "base_url":"http://127.0.0.1:58063",
   [junit4]   2>   "state":"down"} current state version: 3
   [junit4]   2> 1610732 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for [collection1] ver [0]
   [junit4]   2> 1610732 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1610732 INFO  (OverseerStateUpdate-95151506224381956-127.0.0.1:55827_-n_0000000000) [n:127.0.0.1:55827_    ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:58063_",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"3",
   [junit4]   2>   "base_url":"http://127.0.0.1:58063",
   [junit4]   2>   "state":"down"}
   [junit4]   2> 1610732 INFO  (OverseerStateUpdate-95151506224381956-127.0.0.1:55827_-n_0000000000) [n:127.0.0.1:55827_    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=3
   [junit4]   2> 1610732 INFO  (OverseerStateUpdate-95151506224381956-127.0.0.1:55827_-n_0000000000) [n:127.0.0.1:55827_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard3
   [junit4]   2> 1610836 INFO  (OverseerStateUpdate-95151506224381956-127.0.0.1:55827_-n_0000000000) [n:127.0.0.1:55827_    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 1610837 INFO  (zkCallback-1014-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1610838 INFO  (zkCallback-1014-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_    ] o.a.s.c.c.ZkStateReader Updating data for [collection1] from [0] to [1]
   [junit4]   2> 1611732 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1611732 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1611733 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1611733 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/collection1]
   [junit4]   2> 1611734 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/collection1] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1611734 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/shard-1-001/cores/collection1'
   [junit4]   2> 1611734 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1611734 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1611736 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1611745 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1611775 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1611807 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1611815 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1612057 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1612059 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1612060 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1612063 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1612085 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1612086 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 1612086 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1612086 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/shard-1-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/shard-1-001/cores/collection1/data/]
   [junit4]   2> 1612086 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@339b9
   [junit4]   2> 1612087 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/shard-1-001/cores/collection1/data
   [junit4]   2> 1612087 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/shard-1-001/cores/collection1/data/index/
   [junit4]   2> 1612087 WARN  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/shard-1-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1612088 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/shard-1-001/cores/collection1/data/index
   [junit4]   2> 1612088 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@50b58657
   [junit4]   2> 1612088 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@3d86e326 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@62aaf358),segFN=segments_1,generation=1}
   [junit4]   2> 1612088 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1612089 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:58063_ c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/shard-1-001/cores/collection1/data/
   [junit4]   2> 1612089 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:58063_ c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_52BE79E325438F54-001/shard-1-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1612096 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1612096 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1612096 INFO  (coreLoadExecutor-3425-thread-1-processing-n:127.0.0.1:58063_) [n:127.0.0.1:58063_ c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1612096

[...truncated too long message...]

/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-admin-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.2.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20160104_081232_052.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20160104_081232_052.suites -stdin
   [junit4] ERROR: JVM J1 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1444: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1000: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 275 minutes 17 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1064 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1064/

1 tests failed.
FAILED:  org.apache.lucene.util.TestGeoUtils.testGeoRelations

Error Message:
1 incorrect hits (see above)

Stack Trace:
java.lang.AssertionError: 1 incorrect hits (see above)
	at __randomizedtesting.SeedInfo.seed([F3EF2CEF947DF2D7:31CC385AE08A8469]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.lucene.util.TestGeoUtils.testGeoRelations(TestGeoUtils.java:533)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 8498 lines...]
   [junit4] Suite: org.apache.lucene.util.TestGeoUtils
   [junit4]   1> doc=4496 matched but should not on iteration 142
   [junit4]   1>   lon=51.95133626461029 lat=89.46778954938054 distanceMeters=164964.87198676894 vs radiusMeters=164532.3152143511
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestGeoUtils -Dtests.method=testGeoRelations -Dtests.seed=F3EF2CEF947DF2D7 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=el_CY -Dtests.timezone=America/Rankin_Inlet -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] FAILURE 3.58s J2 | TestGeoUtils.testGeoRelations <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: 1 incorrect hits (see above)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F3EF2CEF947DF2D7:31CC385AE08A8469]:0)
   [junit4]    > 	at org.apache.lucene.util.TestGeoUtils.testGeoRelations(TestGeoUtils.java:533)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {}, docValues:{}, sim=DefaultSimilarity, locale=el_CY, timezone=America/Rankin_Inlet
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=117013256,total=352321536
   [junit4]   2> NOTE: All tests run in this JVM: [TestTermAutomatonQuery, TestPayloadSpanUtil, TestSpanRegexQuery, TestDocTermOrdsRangeFilter, TestSlowFuzzyQuery2, TestRegexQuery, TestSlowFuzzyQuery, TestJakartaRegexpCapabilities, TestFieldCacheRangeFilter, TestFieldCacheTermsFilter, TestDocValuesNumbersQuery, TestDocValuesRangeQuery, TestGeoUtils]
   [junit4] Completed [14/20 (1!)] on J2 in 3.63s, 8 tests, 1 failure <<< FAILURES!

[...truncated 50 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:472: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2240: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1444: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1000: There were test failures: 20 suites, 142 tests, 1 failure [seed: F3EF2CEF947DF2D7]

Total time: 149 minutes 36 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1063 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1063/

5 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest

Error Message:
6 threads leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest:     1) Thread[id=14817, name=StoppableSearchThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)         at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)         at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:973)         at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:960)         at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:523)         at org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:190)         at org.apache.solr.common.cloud.ClusterState.hasCollection(ClusterState.java:120)         at org.apache.solr.client.solrj.impl.CloudSolrClient.getCollectionNames(CloudSolrClient.java:1111)         at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:834)         at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)         at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)         at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)         at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958)         at org.apache.solr.cloud.StoppableSearchThread.run(StoppableSearchThread.java:55)    2) Thread[id=14814, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:136)    3) Thread[id=14816, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:136)    4) Thread[id=17180, name=StoppableSearchThread-EventThread, state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)    5) Thread[id=14815, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:136)    6) Thread[id=17179, name=StoppableSearchThread-SendThread(127.0.0.1:56771), state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 6 threads leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest: 
   1) Thread[id=14817, name=StoppableSearchThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)
        at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)
        at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
        at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:973)
        at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:960)
        at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:523)
        at org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:190)
        at org.apache.solr.common.cloud.ClusterState.hasCollection(ClusterState.java:120)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.getCollectionNames(CloudSolrClient.java:1111)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:834)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
        at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)
        at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958)
        at org.apache.solr.cloud.StoppableSearchThread.run(StoppableSearchThread.java:55)
   2) Thread[id=14814, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:136)
   3) Thread[id=14816, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:136)
   4) Thread[id=17180, name=StoppableSearchThread-EventThread, state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
   5) Thread[id=14815, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:136)
   6) Thread[id=17179, name=StoppableSearchThread-SendThread(127.0.0.1:56771), state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
        at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
	at __randomizedtesting.SeedInfo.seed([13D116F8AB60D4FF]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=14814, name=StoppableIndexingThread, state=BLOCKED, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at org.apache.solr.client.solrj.util.ClientUtils.writeVal(ClientUtils.java:164)         at org.apache.solr.client.solrj.util.ClientUtils.writeXML(ClientUtils.java:132)         at org.apache.solr.client.solrj.request.UpdateRequest.writeXML(UpdateRequest.java:413)         at org.apache.solr.client.solrj.request.UpdateRequest.getXML(UpdateRequest.java:329)         at org.apache.solr.client.solrj.request.RequestWriter.getContentStream(RequestWriter.java:67)         at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:95)         at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:105)         at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:356)         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:239)         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)         at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)         at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:167)         at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:177)         at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:116)    2) Thread[id=14817, name=StoppableSearchThread, state=RUNNABLE, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at java.io.BufferedWriter.write(BufferedWriter.java:231)         at java.io.Writer.write(Writer.java:157)         at java.io.BufferedWriter.newLine(BufferedWriter.java:243)         at java.io.PrintStream.newLine(PrintStream.java:544)         at java.io.PrintStream.println(PrintStream.java:824)         at java.lang.Throwable$WrappedPrintStream.println(Throwable.java:748)         at java.lang.Throwable.printEnclosedStackTrace(Throwable.java:697)         at java.lang.Throwable.printStackTrace(Throwable.java:667)         at java.lang.Throwable.printStackTrace(Throwable.java:643)         at java.lang.Throwable.printStackTrace(Throwable.java:634)         at org.apache.solr.cloud.StoppableSearchThread.run(StoppableSearchThread.java:58)    3) Thread[id=14816, name=StoppableIndexingThread, state=BLOCKED, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at java.io.PrintStream.println(PrintStream.java:806)         at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:121)    4) Thread[id=14815, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at org.apache.solr.common.util.XML.escape(XML.java:204)         at org.apache.solr.common.util.XML.escapeCharData(XML.java:77)         at org.apache.solr.common.util.XML.writeXML(XML.java:147)         at org.apache.solr.client.solrj.util.ClientUtils.writeVal(ClientUtils.java:164)         at org.apache.solr.client.solrj.util.ClientUtils.writeXML(ClientUtils.java:132)         at org.apache.solr.client.solrj.request.UpdateRequest.writeXML(UpdateRequest.java:413)         at org.apache.solr.client.solrj.request.UpdateRequest.getXML(UpdateRequest.java:329)         at org.apache.solr.client.solrj.request.RequestWriter.getContentStream(RequestWriter.java:67)         at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:95)         at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:105)         at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:356)         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:239)         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)         at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)         at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:167)         at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:177)         at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:116)    5) Thread[id=17179, name=StoppableSearchThread-SendThread(127.0.0.1:56771), state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=14814, name=StoppableIndexingThread, state=BLOCKED, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at org.apache.solr.client.solrj.util.ClientUtils.writeVal(ClientUtils.java:164)
        at org.apache.solr.client.solrj.util.ClientUtils.writeXML(ClientUtils.java:132)
        at org.apache.solr.client.solrj.request.UpdateRequest.writeXML(UpdateRequest.java:413)
        at org.apache.solr.client.solrj.request.UpdateRequest.getXML(UpdateRequest.java:329)
        at org.apache.solr.client.solrj.request.RequestWriter.getContentStream(RequestWriter.java:67)
        at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:95)
        at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:105)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:356)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:239)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:167)
        at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:177)
        at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:116)
   2) Thread[id=14817, name=StoppableSearchThread, state=RUNNABLE, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at java.io.BufferedWriter.write(BufferedWriter.java:231)
        at java.io.Writer.write(Writer.java:157)
        at java.io.BufferedWriter.newLine(BufferedWriter.java:243)
        at java.io.PrintStream.newLine(PrintStream.java:544)
        at java.io.PrintStream.println(PrintStream.java:824)
        at java.lang.Throwable$WrappedPrintStream.println(Throwable.java:748)
        at java.lang.Throwable.printEnclosedStackTrace(Throwable.java:697)
        at java.lang.Throwable.printStackTrace(Throwable.java:667)
        at java.lang.Throwable.printStackTrace(Throwable.java:643)
        at java.lang.Throwable.printStackTrace(Throwable.java:634)
        at org.apache.solr.cloud.StoppableSearchThread.run(StoppableSearchThread.java:58)
   3) Thread[id=14816, name=StoppableIndexingThread, state=BLOCKED, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at java.io.PrintStream.println(PrintStream.java:806)
        at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:121)
   4) Thread[id=14815, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at org.apache.solr.common.util.XML.escape(XML.java:204)
        at org.apache.solr.common.util.XML.escapeCharData(XML.java:77)
        at org.apache.solr.common.util.XML.writeXML(XML.java:147)
        at org.apache.solr.client.solrj.util.ClientUtils.writeVal(ClientUtils.java:164)
        at org.apache.solr.client.solrj.util.ClientUtils.writeXML(ClientUtils.java:132)
        at org.apache.solr.client.solrj.request.UpdateRequest.writeXML(UpdateRequest.java:413)
        at org.apache.solr.client.solrj.request.UpdateRequest.getXML(UpdateRequest.java:329)
        at org.apache.solr.client.solrj.request.RequestWriter.getContentStream(RequestWriter.java:67)
        at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:95)
        at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:105)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:356)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:239)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:167)
        at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:177)
        at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:116)
   5) Thread[id=17179, name=StoppableSearchThread-SendThread(127.0.0.1:56771), state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
        at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
	at __randomizedtesting.SeedInfo.seed([13D116F8AB60D4FF]:0)


FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
There are still nodes recoverying - waited for 330 seconds

Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
	at __randomizedtesting.SeedInfo.seed([13D116F8AB60D4FF:CD5B788590C2B60C]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:175)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:136)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:831)
	at org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR(LeaderInitiatedRecoveryOnShardRestartTest.java:159)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:965)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:940)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test

Error Message:
The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!

Stack Trace:
java.lang.AssertionError: The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
	at __randomizedtesting.SeedInfo.seed([13D116F8AB60D4FF:9B852922059CB907]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.ChaosMonkey.stopTheMonkey(ChaosMonkey.java:560)
	at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test(ChaosMonkeyNothingIsSafeTest.java:186)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:965)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:940)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=10184, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=10184, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
	at __randomizedtesting.SeedInfo.seed([13D116F8AB60D4FF:9B852922059CB907]:0)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:55518: Could not find collection : awholynewstresscollection_collection1_3
	at __randomizedtesting.SeedInfo.seed([13D116F8AB60D4FF]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)




Build Log:
[...truncated 10621 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_13D116F8AB60D4FF-001/init-core-data-001
   [junit4]   2> 744760 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[13D116F8AB60D4FF]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /h/
   [junit4]   2> 744766 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 744767 INFO  (Thread-3150) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 744767 INFO  (Thread-3150) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 744867 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.ZkTestServer start zk server on port:48043
   [junit4]   2> 744867 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 744868 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 744871 INFO  (zkCallback-343-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7537741 name:ZooKeeperConnection Watcher:127.0.0.1:48043 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 744871 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 744872 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 744872 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 744875 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 744875 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x152016298210000, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 744876 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 744883 INFO  (zkCallback-344-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@11dcdc5d name:ZooKeeperConnection Watcher:127.0.0.1:48043/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 744883 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 744883 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 744883 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 744886 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 744889 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 744893 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 744896 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 744897 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 744901 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 744901 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 744904 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 744904 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 744907 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 744908 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 744909 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 744910 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 744912 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 744912 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 744914 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 744914 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 744916 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 744916 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 744919 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 744919 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 744921 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 744921 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 744925 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 744925 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 745259 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_13D116F8AB60D4FF-001/control-001/cores/collection1
   [junit4]   2> 745261 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 745263 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@19ff856c{/h,null,AVAILABLE}
   [junit4]   2> 745264 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.e.j.s.ServerConnector Started ServerConnector@569082c9{HTTP/1.1}{127.0.0.1:48377}
   [junit4]   2> 745264 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.e.j.s.Server Started @748238ms
   [junit4]   2> 745264 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_13D116F8AB60D4FF-001/tempDir-001/control/data, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_13D116F8AB60D4FF-001/control-001/cores, hostContext=/h, hostPort=48377}
   [junit4]   2> 745264 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 745264 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_13D116F8AB60D4FF-001/control-001'
   [junit4]   2> 745264 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 745265 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 745265 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 745265 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 745268 INFO  (zkCallback-345-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@44c9a340 name:ZooKeeperConnection Watcher:127.0.0.1:48043/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 745269 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 745269 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 745270 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 745270 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_13D116F8AB60D4FF-001/control-001/solr.xml
   [junit4]   2> 745285 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_13D116F8AB60D4FF-001/control-001/cores
   [junit4]   2> 745286 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.CoreContainer New CoreContainer 1341190191
   [junit4]   2> 745286 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_13D116F8AB60D4FF-001/control-001]
   [junit4]   2> 745286 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_13D116F8AB60D4FF-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_13D116F8AB60D4FF-001/control-001/lib
   [junit4]   2> 745286 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 745287 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 745288 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 745288 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 745288 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48043/solr
   [junit4]   2> 745288 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 745288 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 745289 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 745290 INFO  (zkCallback-347-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3c67fe28 name:ZooKeeperConnection Watcher:127.0.0.1:48043 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 745291 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 745291 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 745294 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [n:127.0.0.1:48377_h    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 745296 INFO  (zkCallback-348-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@253136c4 name:ZooKeeperConnection Watcher:127.0.0.1:48043/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 745296 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [n:127.0.0.1:48377_h    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 745299 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [n:127.0.0.1:48377_h    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 745302 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [n:127.0.0.1:48377_h    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 745306 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [n:127.0.0.1:48377_h    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 745310 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [n:127.0.0.1:48377_h    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 745312 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [n:127.0.0.1:48377_h    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 745315 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [n:127.0.0.1:48377_h    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 745318 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [n:127.0.0.1:48377_h    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 745320 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [n:127.0.0.1:48377_h    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 745322 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [n:127.0.0.1:48377_h    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 745323 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [n:127.0.0.1:48377_h    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48377_h
   [junit4]   2> 745324 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [n:127.0.0.1:48377_h    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:48377_h
   [junit4]   2> 745326 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [n:127.0.0.1:48377_h    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 745327 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [n:127.0.0.1:48377_h    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 745328 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [n:127.0.0.1:48377_h    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 745329 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [n:127.0.0.1:48377_h    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95140065098924036-127.0.0.1:48377_h-n_0000000000
   [junit4]   2> 745330 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [n:127.0.0.1:48377_h    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:48377_h
   [junit4]   2> 745330 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [n:127.0.0.1:48377_h    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 745331 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [n:127.0.0.1:48377_h    ] o.a.s.c.Overseer Overseer (id=95140065098924036-127.0.0.1:48377_h-n_0000000000) starting
   [junit4]   2> 745333 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [n:127.0.0.1:48377_h    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 745341 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [n:127.0.0.1:48377_h    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 745342 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [n:127.0.0.1:48377_h    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 745342 INFO  (OverseerStateUpdate-95140065098924036-127.0.0.1:48377_h-n_0000000000) [n:127.0.0.1:48377_h    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 745343 INFO  (OverseerCollectionConfigSetProcessor-95140065098924036-127.0.0.1:48377_h-n_0000000000) [n:127.0.0.1:48377_h    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 745359 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [n:127.0.0.1:48377_h    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 745360 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [n:127.0.0.1:48377_h    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 745361 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [n:127.0.0.1:48377_h    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_13D116F8AB60D4FF-001/control-001/cores
   [junit4]   2> 745362 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [n:127.0.0.1:48377_h    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {loadOnStartup=true, config=solrconfig.xml, transient=false, collection=control_collection, configSetProperties=configsetprops.json, coreNodeName=, schema=schema.xml, dataDir=data/, name=collection1, shard=}
   [junit4]   2> 745362 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [n:127.0.0.1:48377_h    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_13D116F8AB60D4FF-001/control-001/cores/collection1
   [junit4]   2> 745362 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [n:127.0.0.1:48377_h    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 745364 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [n:127.0.0.1:48377_h    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 745364 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [n:127.0.0.1:48377_h    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 745364 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 745364 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 745366 INFO  (OverseerStateUpdate-95140065098924036-127.0.0.1:48377_h-n_0000000000) [n:127.0.0.1:48377_h    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:48377/h",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:48377_h",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null} current state version: 0
   [junit4]   2> 745367 INFO  (OverseerStateUpdate-95140065098924036-127.0.0.1:48377_h-n_0000000000) [n:127.0.0.1:48377_h    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:48377/h",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:48377_h",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null}
   [junit4]   2> 745367 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 745367 INFO  (OverseerStateUpdate-95140065098924036-127.0.0.1:48377_h-n_0000000000) [n:127.0.0.1:48377_h    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 745368 INFO  (OverseerStateUpdate-95140065098924036-127.0.0.1:48377_h-n_0000000000) [n:127.0.0.1:48377_h    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 745370 INFO  (zkCallback-348-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 746368 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 746368 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 746368 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 746368 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 746369 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 746370 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_13D116F8AB60D4FF-001/control-001/cores/collection1'
   [junit4]   2> 746370 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 746370 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 746373 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 746382 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 746405 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 746439 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 746450 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 746741 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 746750 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 746752 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 746766 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 746770 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 746773 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 746774 WARN  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 746774 WARN  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 746774 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 746776 WARN  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 746776 WARN  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 746777 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 746777 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 746778 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore solr.StandardDirectoryFactory
   [junit4]   2> 746778 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_13D116F8AB60D4FF-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_13D116F8AB60D4FF-001/control-001/cores/collection1/data/]
   [junit4]   2> 746778 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@187b2e64
   [junit4]   2> 746779 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_13D116F8AB60D4FF-001/control-001/cores/collection1/data
   [junit4]   2> 746779 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_13D116F8AB60D4FF-001/control-001/cores/collection1/data/index/
   [junit4]   2> 746779 WARN  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_13D116F8AB60D4FF-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 746780 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_13D116F8AB60D4FF-001/control-001/cores/collection1/data/index
   [junit4]   2> 746780 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@702beb0b
   [junit4]   2> 746809 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_13D116F8AB60D4FF-001/control-001/cores/collection1/data/index,segFN=segments_1,generation=1}
   [junit4]   2> 746809 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 746810 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_13D116F8AB60D4FF-001/control-001/cores/collection1/data/
   [junit4]   2> 746817 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 746818 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 746818 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 746818 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 746818 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 746818 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 746819 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 746819 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 746819 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 746819 WARN  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 746822 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 746825 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 746827 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 746829 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 746844 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: standard,/schema,/dump,/admin/mbeans,/update/json,/admin/logging,/get,/admin/threads,/admin/plugins,/admin/properties,/update,/admin/file,/replication,/update/csv,/config,/update/json/docs,/admin/ping,/admin/segments,/admin/luke,/admin/system
   [junit4]   2> 746845 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 746846 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 746846 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 746846 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 746847 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 746847 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=44, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5509535939053533]
   [junit4]   2> 746848 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_13D116F8AB60D4FF-001/control-001/cores/collection1/data/index,segFN=segments_1,generation=1}
   [junit4]   2> 746848 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 746848 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@2e506753[collection1] main
   [junit4]   2> 746848 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 746849 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 746849 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 746850 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 746850 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 746851 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 746851 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 746851 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 746851 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 746851 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 746852 INFO  (searcherExecutor-1088-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2e506753[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 746853 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 746853 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 746853 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 746853 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1522241043768016896
   [junit4]   2> 746855 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1522241043768016896
   [junit4]   2> 746855 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 746856 INFO  (coreLoadExecutor-1087-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 746857 INFO  (coreZkRegister-1081-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:48377/h collection:control_collection shard:shard1
   [junit4]   2> 746858 INFO  (coreZkRegister-1081-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 746862 INFO  (coreZkRegister-1081-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95140065098924036-core_node1-n_0000000000
   [junit4]   2> 746862 INFO  (coreZkRegister-1081-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=300000
   [junit4]   2> 746866 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 746869 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 746870 INFO  (zkCallback-350-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@afb146c name:ZooKeeperConnection Watcher:127.0.0.1:48043/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 746870 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 746870 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 746870 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 746871 INFO  (coreZkRegister-1081-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 746871 INFO  (coreZkRegister-1081-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 746871 INFO  (coreZkRegister-1081-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:48377/h/collection1/
   [junit4]   2> 746871 INFO  (coreZkRegister-1081-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 746871 INFO  (coreZkRegister-1081-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:48377/h/collection1/ has no replicas
   [junit4]   2> 746872 INFO  (OverseerStateUpdate-95140065098924036-127.0.0.1:48377_h-n_0000000000) [n:127.0.0.1:48377_h    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 746873 INFO  (coreZkRegister-1081-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 746874 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[13D116F8AB60D4FF]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 746874 INFO  (zkCallback-350-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 746874 INFO  (zkCallback-348-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 746879 INFO  (coreZkRegister-1081-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 746881 INFO  (coreZkRegister-1081-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:48377/h/collection1/ shard1
   [junit4]   2> 746881 INFO  (OverseerStateUpdate-95140065098924036-127.0.0.1:48377_h-n_0000000000) [n:127.0.0.1:48377_h    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:48377/h",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 746984 INFO  (zkCallback-348-thread-1-processing-n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 746984 INFO  (zkCallback-350-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 747032 INFO  (coreZkRegister-1081-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:48377/h/collection1/ and leader is http://127.0.0.1:48377/h/collection1/
   [junit4]   2> 747032 INFO  (coreZkRegister-1081-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:48377/h
   [junit4]   2> 747033 INFO  (coreZkRegister-1081-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:48377_h) [n:127.0.0.1:48377_h c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 747033 I

[...truncated too long message...]

che.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:834)
   [junit4]    >         at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
   [junit4]    >         at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
   [junit4]    >         at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)
   [junit4]    >         at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958)
   [junit4]    >         at org.apache.solr.cloud.StoppableSearchThread.run(StoppableSearchThread.java:55)
   [junit4]    >    2) Thread[id=14814, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:136)
   [junit4]    >    3) Thread[id=14816, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:136)
   [junit4]    >    4) Thread[id=17180, name=StoppableSearchThread-EventThread, state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
   [junit4]    >    5) Thread[id=14815, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:136)
   [junit4]    >    6) Thread[id=17179, name=StoppableSearchThread-SendThread(127.0.0.1:56771), state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([13D116F8AB60D4FF]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=14814, name=StoppableIndexingThread, state=BLOCKED, group=TGRP-ChaosMonkeyNothingIsSafeTest]
   [junit4]    >         at org.apache.solr.client.solrj.util.ClientUtils.writeVal(ClientUtils.java:164)
   [junit4]    >         at org.apache.solr.client.solrj.util.ClientUtils.writeXML(ClientUtils.java:132)
   [junit4]    >         at org.apache.solr.client.solrj.request.UpdateRequest.writeXML(UpdateRequest.java:413)
   [junit4]    >         at org.apache.solr.client.solrj.request.UpdateRequest.getXML(UpdateRequest.java:329)
   [junit4]    >         at org.apache.solr.client.solrj.request.RequestWriter.getContentStream(RequestWriter.java:67)
   [junit4]    >         at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:95)
   [junit4]    >         at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:105)
   [junit4]    >         at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:356)
   [junit4]    >         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:239)
   [junit4]    >         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    >         at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
   [junit4]    >         at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:167)
   [junit4]    >         at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:177)
   [junit4]    >         at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:116)
   [junit4]    >    2) Thread[id=14817, name=StoppableSearchThread, state=RUNNABLE, group=TGRP-ChaosMonkeyNothingIsSafeTest]
   [junit4]    >         at java.io.BufferedWriter.write(BufferedWriter.java:231)
   [junit4]    >         at java.io.Writer.write(Writer.java:157)
   [junit4]    >         at java.io.BufferedWriter.newLine(BufferedWriter.java:243)
   [junit4]    >         at java.io.PrintStream.newLine(PrintStream.java:544)
   [junit4]    >         at java.io.PrintStream.println(PrintStream.java:824)
   [junit4]    >         at java.lang.Throwable$WrappedPrintStream.println(Throwable.java:748)
   [junit4]    >         at java.lang.Throwable.printEnclosedStackTrace(Throwable.java:697)
   [junit4]    >         at java.lang.Throwable.printStackTrace(Throwable.java:667)
   [junit4]    >         at java.lang.Throwable.printStackTrace(Throwable.java:643)
   [junit4]    >         at java.lang.Throwable.printStackTrace(Throwable.java:634)
   [junit4]    >         at org.apache.solr.cloud.StoppableSearchThread.run(StoppableSearchThread.java:58)
   [junit4]    >    3) Thread[id=14816, name=StoppableIndexingThread, state=BLOCKED, group=TGRP-ChaosMonkeyNothingIsSafeTest]
   [junit4]    >         at java.io.PrintStream.println(PrintStream.java:806)
   [junit4]    >         at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:121)
   [junit4]    >    4) Thread[id=14815, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-ChaosMonkeyNothingIsSafeTest]
   [junit4]    >         at org.apache.solr.common.util.XML.escape(XML.java:204)
   [junit4]    >         at org.apache.solr.common.util.XML.escapeCharData(XML.java:77)
   [junit4]    >         at org.apache.solr.common.util.XML.writeXML(XML.java:147)
   [junit4]    >         at org.apache.solr.client.solrj.util.ClientUtils.writeVal(ClientUtils.java:164)
   [junit4]    >         at org.apache.solr.client.solrj.util.ClientUtils.writeXML(ClientUtils.java:132)
   [junit4]    >         at org.apache.solr.client.solrj.request.UpdateRequest.writeXML(UpdateRequest.java:413)
   [junit4]    >         at org.apache.solr.client.solrj.request.UpdateRequest.getXML(UpdateRequest.java:329)
   [junit4]    >         at org.apache.solr.client.solrj.request.RequestWriter.getContentStream(RequestWriter.java:67)
   [junit4]    >         at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:95)
   [junit4]    >         at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:105)
   [junit4]    >         at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:356)
   [junit4]    >         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:239)
   [junit4]    >         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    >         at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
   [junit4]    >         at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:167)
   [junit4]    >         at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:177)
   [junit4]    >         at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:116)
   [junit4]    >    5) Thread[id=17179, name=StoppableSearchThread-SendThread(127.0.0.1:56771), state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([13D116F8AB60D4FF]:0)
   [junit4] Completed [418/562 (3!)] on J1 in 199.57s, 1 test, 1 failure, 2 errors <<< FAILURES!

[...truncated 454 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1444: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1000: There were test failures: 562 suites (7 ignored), 2231 tests, 2 suite-level errors, 1 error, 2 failures, 423 ignored (11 assumptions) [seed: 13D116F8AB60D4FF]

Total time: 190 minutes 25 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1062 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1062/

3 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at https://127.0.0.1:42409/kbb/z: non ok status: 500, message:Server Error

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:42409/kbb/z: non ok status: 500, message:Server Error
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:508)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:295)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:412)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:162)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:965)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:940)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
86 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest:     1) Thread[id=23643, name=searcherExecutor-4982-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=23811, name=zkCallback-1310-thread-12-processing-n:127.0.0.1:42409_kbb%2Fz-SendThread(127.0.0.1:59624), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)    3) Thread[id=23792, name=zkCallback-1310-thread-12-processing-n:127.0.0.1:42409_kbb%2Fz, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Object.wait(Native Method)         at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:201)         at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:139)         at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)         at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:133)         at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:264)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:232)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=22059, name=searcherExecutor-4034-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=23410, name=searcherExecutor-4823-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=23622, name=searcherExecutor-4965-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    7) Thread[id=23458, name=searcherExecutor-4865-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    8) Thread[id=23393, name=searcherExecutor-4766-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    9) Thread[id=23499, name=searcherExecutor-4888-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   10) Thread[id=22333, name=searcherExecutor-4220-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   11) Thread[id=22129, name=searcherExecutor-4103-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   12) Thread[id=23406, name=searcherExecutor-4787-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   13) Thread[id=23381, name=searcherExecutor-4763-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   14) Thread[id=22359, name=searcherExecutor-4240-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   15) Thread[id=21996, name=searcherExecutor-3988-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   16) Thread[id=23648, name=searcherExecutor-4972-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   17) Thread[id=23644, name=searcherExecutor-5018-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   18) Thread[id=23412, name=searcherExecutor-4827-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   19) Thread[id=23520, name=searcherExecutor-4906-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   20) Thread[id=23356, name=searcherExecutor-4746-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   21) Thread[id=23800, name=zkCallback-1314-thread-20-processing-n:127.0.0.1:47772_kbb%2Fz-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)   22) Thread[id=23621, name=searcherExecutor-4973-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   23) Thread[id=23351, name=searcherExecutor-4742-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   24) Thread[id=23525, name=searcherExecutor-4899-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   25) Thread[id=21998, name=searcherExecutor-3996-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   26) Thread[id=22128, name=searcherExecutor-4102-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   27) Thread[id=23562, name=searcherExecutor-4926-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   28) Thread[id=23404, name=searcherExecutor-4783-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   29) Thread[id=23808, name=zkCallback-1304-thread-22-processing-n:127.0.0.1:56729_kbb%2Fz-SendThread(127.0.0.1:59624), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)   30) Thread[id=23566, name=searcherExecutor-4930-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   31) Thread[id=23497, name=searcherExecutor-4877-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   32) Thread[id=23810, name=zkCallback-1304-thread-22-processing-n:127.0.0.1:56729_kbb%2Fz-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)   33) Thread[id=22099, name=searcherExecutor-4066-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   34) Thread[id=21873, name=searcherExecutor-3905-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   35) Thread[id=23462, name=searcherExecutor-4860-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   36) Thread[id=23445, name=searcherExecutor-4836-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   37) Thread[id=23501, name=searcherExecutor-4857-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   38) Thread[id=23652, name=searcherExecutor-5001-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   39) Thread[id=22046, name=searcherExecutor-4025-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   40) Thread[id=23609, name=searcherExecutor-4959-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   41) Thread[id=21878, name=searcherExecutor-3911-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   42) Thread[id=21990, name=searcherExecutor-3970-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   43) Thread[id=23770, name=zkCallback-1304-thread-22-processing-n:127.0.0.1:56729_kbb%2Fz, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Object.wait(Native Method)         at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:201)         at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:139)         at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)         at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:133)         at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:264)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:232)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   44) Thread[id=21965, name=searcherExecutor-3952-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   45) Thread[id=21992, name=searcherExecutor-3974-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   46) Thread[id=22098, name=searcherExecutor-4068-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   47) Thread[id=23376, name=searcherExecutor-4751-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   48) Thread[id=21766, name=searcherExecutor-3862-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   49) Thread[id=23396, name=searcherExecutor-4807-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   50) Thread[id=23812, name=zkCallback-1310-thread-12-processing-n:127.0.0.1:42409_kbb%2Fz-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)   51) Thread[id=23640, name=searcherExecutor-5036-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   52) Thread[id=22353, name=searcherExecutor-4230-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   53) Thread[id=21794, name=searcherExecutor-3873-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   54) Thread[id=21995, name=searcherExecutor-3972-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   55) Thread[id=23764, name=zkCallback-1314-thread-20-processing-n:127.0.0.1:47772_kbb%2Fz, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Object.wait(Native Method)         at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:201)         at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:139)         at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)         at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:133)         at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:264)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:232)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   56) Thread[id=23506, name=searcherExecutor-4895-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   57) Thread[id=23446, name=searcherExecutor-4850-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   58) Thread[id=23407, name=searcherExecutor-4800-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   59) Thread[id=23425, name=searcherExecutor-4808-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   60) Thread[id=22053, name=searcherExecutor-4030-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   61) Thread[id=23371, name=searcherExecutor-4764-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   62) Thread[id=21735, name=searcherExecutor-3851-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   63) Thread[id=22066, name=searcherExecutor-4056-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   64) Thread[id=23614, name=searcherExecutor-4962-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   65) Thread[id=23651, name=searcherExecutor-5027-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   66) Thread[id=23388, name=searcherExecutor-4759-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   67) Thread[id=21869, name=searcherExecutor-3900-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   68) Thread[id=22104, name=searcherExecutor-4067-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   69) Thread[id=23450, name=searcherExecutor-4858-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   70) Thread[id=23378, name=searcherExecutor-4758-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   71) Thread[id=22125, name=searcherExecutor-4098-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   72) Thread[id=23296, name=searcherExecutor-4740-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   73) Thread[id=23494, name=searcherExecutor-4882-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   74) Thread[id=23564, name=searcherExecutor-4937-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   75) Thread[id=23619, name=searcherExecutor-4968-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   76) Thread[id=23375, name=searcherExecutor-4768-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   77) Thread[id=23569, name=searcherExecutor-4944-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   78) Thread[id=23799, name=zkCallback-1314-thread-20-processing-n:127.0.0.1:47772_kbb%2Fz-SendThread(127.0.0.1:59624), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)   79) Thread[id=23615, name=searcherExecutor-4967-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   80) Thread[id=21994, name=searcherExecutor-3976-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   81) Thread[id=22167, name=searcherExecutor-4125-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   82) Thread[id=21989, name=searcherExecutor-3973-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   83) Thread[id=22065, name=searcherExecutor-4055-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   84) Thread[id=21986, name=searcherExecutor-3968-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   85) Thread[id=22101, name=searcherExecutor-4065-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   86) Thread[id=23576, name=searcherExecutor-4943-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 86 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest: 
   1) Thread[id=23643, name=searcherExecutor-4982-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=23811, name=zkCallback-1310-thread-12-processing-n:127.0.0.1:42409_kbb%2Fz-SendThread(127.0.0.1:59624), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
   3) Thread[id=23792, name=zkCallback-1310-thread-12-processing-n:127.0.0.1:42409_kbb%2Fz, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at java.lang.Object.wait(Native Method)
        at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:201)
        at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:139)
        at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
        at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:133)
        at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:264)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:232)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=22059, name=searcherExecutor-4034-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=23410, name=searcherExecutor-4823-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   6) Thread[id=23622, name=searcherExecutor-4965-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   7) Thread[id=23458, name=searcherExecutor-4865-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   8) Thread[id=23393, name=searcherExecutor-4766-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   9) Thread[id=23499, name=searcherExecutor-4888-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  10) Thread[id=22333, name=searcherExecutor-4220-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  11) Thread[id=22129, name=searcherExecutor-4103-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  12) Thread[id=23406, name=searcherExecutor-4787-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  13) Thread[id=23381, name=searcherExecutor-4763-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  14) Thread[id=22359, name=searcherExecutor-4240-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  15) Thread[id=21996, name=searcherExecutor-3988-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  16) Thread[id=23648, name=searcherExecutor-4972-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  17) Thread[id=23644, name=searcherExecutor-5018-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  18) Thread[id=23412, name=searcherExecutor-4827-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  19) Thread[id=23520, name=searcherExecutor-4906-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  20) Thread[id=23356, name=searcherExecutor-4746-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  21) Thread[id=23800, name=zkCallback-1314-thread-20-processing-n:127.0.0.1:47772_kbb%2Fz-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
  22) Thread[id=23621, name=searcherExecutor-4973-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  23) Thread[id=23351, name=searcherExecutor-4742-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  24) Thread[id=23525, name=searcherExecutor-4899-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
  25) Thread[id=21998, name=searcherExecutor-3996-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java

[...truncated too long message...]

take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   73) Thread[id=23497, name=searcherExecutor-4877-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   74) Thread[id=23770, name=zkCallback-1304-thread-22, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   75) Thread[id=21965, name=searcherExecutor-3952-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   76) Thread[id=21992, name=searcherExecutor-3974-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   77) Thread[id=22098, name=searcherExecutor-4068-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   78) Thread[id=23609, name=searcherExecutor-4959-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   79) Thread[id=21878, name=searcherExecutor-3911-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   80) Thread[id=21990, name=searcherExecutor-3970-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([FBD036411D7C6F21]:0)
   [junit4] Completed [253/562 (1!)] on J1 in 616.68s, 1 test, 3 errors <<< FAILURES!

[...truncated 825 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151231_061649_580.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid25711.hprof ...
   [junit4] Heap dump file created [612842516 bytes in 7.894 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 125 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1444: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1000: There were test failures: 562 suites (7 ignored), 2231 tests, 2 suite-level errors, 1 error, 1122 ignored (9 assumptions) [seed: FBD036411D7C6F21]

Total time: 262 minutes 21 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



Re: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1061 - Still Failing

Posted by Mark Miller <ma...@gmail.com>.
I'll raise the time on this for nightly runs.

- Mark

On Tue, Dec 29, 2015 at 6:47 PM Apache Jenkins Server <
jenkins@builds.apache.org> wrote:

> Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1061/
>
> 3 tests failed.
> FAILED:  org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest.test
>
> Error Message:
> The Monkey ran for over 45 seconds and no jetties were stopped - this is
> worth investigating!
>
> Stack Trace:
> java.lang.AssertionError: The Monkey ran for over 45 seconds and no
> jetties were stopped - this is worth investigating!
>         at
> __randomizedtesting.SeedInfo.seed([EA3B02E14E63D573:626F3D3BE09FB88B]:0)
>         at org.junit.Assert.fail(Assert.java:93)
>         at
> org.apache.solr.cloud.ChaosMonkey.stopTheMonkey(ChaosMonkey.java:560)
>         at
> org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test(ChaosMonkeySafeLeaderTest.java:146)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
>         at
> org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:965)
>         at
> org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:940)
>         at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
>         at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>         at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>         at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>         at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>         at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
>         at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>         at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>         at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>         at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>         at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>         at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
>         at java.lang.Thread.run(Thread.java:745)
>
>
> FAILED:  junit.framework.TestSuite.org
> .apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest
>
> Error Message:
> 2 threads leaked from SUITE scope at
> org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest:     1)
> Thread[id=15765, name=StoppableIndexingThread, state=TIMED_WAITING,
> group=TGRP-HdfsChaosMonkeySafeLeaderTest]         at
> java.lang.Thread.sleep(Native Method)         at
> org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:136)
>   2) Thread[id=15764, name=StoppableIndexingThread, state=TIMED_WAITING,
> group=TGRP-HdfsChaosMonkeySafeLeaderTest]         at
> java.lang.Thread.sleep(Native Method)         at
> org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:136)
>
> Stack Trace:
> com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from
> SUITE scope at org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest:
>    1) Thread[id=15765, name=StoppableIndexingThread, state=TIMED_WAITING,
> group=TGRP-HdfsChaosMonkeySafeLeaderTest]
>         at java.lang.Thread.sleep(Native Method)
>         at
> org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:136)
>    2) Thread[id=15764, name=StoppableIndexingThread, state=TIMED_WAITING,
> group=TGRP-HdfsChaosMonkeySafeLeaderTest]
>         at java.lang.Thread.sleep(Native Method)
>         at
> org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:136)
>         at __randomizedtesting.SeedInfo.seed([EA3B02E14E63D573]:0)
>
>
> FAILED:  junit.framework.TestSuite.org
> .apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest
>
> Error Message:
> There are still zombie threads that couldn't be terminated:    1)
> Thread[id=15764, name=StoppableIndexingThread, state=RUNNABLE,
> group=TGRP-HdfsChaosMonkeySafeLeaderTest]         at
> java.lang.Integer.getChars(Integer.java:372)         at
> java.lang.Integer.toString(Integer.java:332)         at
> java.lang.Integer.toString(Integer.java:739)         at
> org.apache.solr.client.solrj.util.ClientUtils.writeVal(ClientUtils.java:164)
>        at
> org.apache.solr.client.solrj.util.ClientUtils.writeXML(ClientUtils.java:132)
>        at
> org.apache.solr.client.solrj.request.UpdateRequest.writeXML(UpdateRequest.java:413)
>        at
> org.apache.solr.client.solrj.request.UpdateRequest.getXML(UpdateRequest.java:329)
>        at
> org.apache.solr.client.solrj.request.RequestWriter.getContentStream(RequestWriter.java:67)
>        at
> org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:95)
>        at
> org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:105)
>        at
> org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:356)
>        at
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:239)
>        at
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
>        at
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
>    at
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:167)
>    at
> org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:177)
>        at
> org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:116)
>   2) Thread[id=15765, name=StoppableIndexingThread, state=RUNNABLE,
> group=TGRP-HdfsChaosMonkeySafeLeaderTest]         at
> java.lang.Integer.getChars(Integer.java:372)         at
> java.lang.Integer.toString(Integer.java:332)         at
> java.lang.Integer.toString(Integer.java:739)         at
> org.apache.solr.client.solrj.util.ClientUtils.writeVal(ClientUtils.java:164)
>        at
> org.apache.solr.client.solrj.util.ClientUtils.writeXML(ClientUtils.java:132)
>        at
> org.apache.solr.client.solrj.request.UpdateRequest.writeXML(UpdateRequest.java:413)
>        at
> org.apache.solr.client.solrj.request.UpdateRequest.getXML(UpdateRequest.java:329)
>        at
> org.apache.solr.client.solrj.request.RequestWriter.getContentStream(RequestWriter.java:67)
>        at
> org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:95)
>        at
> org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:105)
>        at
> org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:356)
>        at
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:239)
>        at
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
>        at
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
>    at
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:167)
>    at
> org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:177)
>        at
> org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:116)
>
> Stack Trace:
> com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie
> threads that couldn't be terminated:
>    1) Thread[id=15764, name=StoppableIndexingThread, state=RUNNABLE,
> group=TGRP-HdfsChaosMonkeySafeLeaderTest]
>         at java.lang.Integer.getChars(Integer.java:372)
>         at java.lang.Integer.toString(Integer.java:332)
>         at java.lang.Integer.toString(Integer.java:739)
>         at
> org.apache.solr.client.solrj.util.ClientUtils.writeVal(ClientUtils.java:164)
>         at
> org.apache.solr.client.solrj.util.ClientUtils.writeXML(ClientUtils.java:132)
>         at
> org.apache.solr.client.solrj.request.UpdateRequest.writeXML(UpdateRequest.java:413)
>         at
> org.apache.solr.client.solrj.request.UpdateRequest.getXML(UpdateRequest.java:329)
>         at
> org.apache.solr.client.solrj.request.RequestWriter.getContentStream(RequestWriter.java:67)
>         at
> org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:95)
>         at
> org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:105)
>         at
> org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:356)
>         at
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:239)
>         at
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
>         at
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
>         at
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:167)
>         at
> org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:177)
>         at
> org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:116)
>    2) Thread[id=15765, name=StoppableIndexingThread, state=RUNNABLE,
> group=TGRP-HdfsChaosMonkeySafeLeaderTest]
>         at java.lang.Integer.getChars(Integer.java:372)
>         at java.lang.Integer.toString(Integer.java:332)
>         at java.lang.Integer.toString(Integer.java:739)
>         at
> org.apache.solr.client.solrj.util.ClientUtils.writeVal(ClientUtils.java:164)
>         at
> org.apache.solr.client.solrj.util.ClientUtils.writeXML(ClientUtils.java:132)
>         at
> org.apache.solr.client.solrj.request.UpdateRequest.writeXML(UpdateRequest.java:413)
>         at
> org.apache.solr.client.solrj.request.UpdateRequest.getXML(UpdateRequest.java:329)
>         at
> org.apache.solr.client.solrj.request.RequestWriter.getContentStream(RequestWriter.java:67)
>         at
> org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:95)
>         at
> org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:105)
>         at
> org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:356)
>         at
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:239)
>         at
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
>         at
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
>         at
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:167)
>         at
> org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:177)
>         at
> org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:116)
>         at __randomizedtesting.SeedInfo.seed([EA3B02E14E63D573]:0)
>
>
>
>
> Build Log:
> [...truncated 11542 lines...]
>    [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest
>    [junit4]   2> Creating dataDir:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/init-core-data-001
>    [junit4]   2> 3024380 INFO
> (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ]
> o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true)
>    [junit4]   2> 3024380 INFO
> (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ]
> o.a.s.BaseDistributedSearchTestCase Setting hostContext system property:
> /rz/r
>    [junit4]   1> Formatting using clusterid: testClusterID
>    [junit4]   2> 3024425 WARN
> (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ]
> o.a.h.m.i.MetricsConfig Cannot locate configuration: tried
> hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
>    [junit4]   2> 3024434 WARN
> (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ]
> o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
>    [junit4]   2> 3024437 INFO
> (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ]
> o.m.log jetty-6.1.26
>    [junit4]   2> 3024450 INFO
> (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ]
> o.m.log Extract
> jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs
> to ./temp/Jetty_localhost_40848_hdfs____.j54i3a/webapp
>    [junit4]   2> 3024574 INFO
> (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ]
> o.m.log NO JSP Support for /, did not find
> org.apache.jasper.servlet.JspServlet
>    [junit4]   2> 3024791 INFO
> (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ]
> o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost
> :40848
>    [junit4]   2> 3024870 WARN
> (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ]
> o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
>    [junit4]   2> 3024871 INFO
> (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ]
> o.m.log jetty-6.1.26
>    [junit4]   2> 3024886 INFO
> (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ]
> o.m.log Extract
> jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode
> to ./temp/Jetty_localhost_58802_datanode____fny2am/webapp
>    [junit4]   2> 3025001 INFO
> (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ]
> o.m.log NO JSP Support for /, did not find
> org.apache.jasper.servlet.JspServlet
>    [junit4]   2> 3025322 INFO
> (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ]
> o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost
> :58802
>    [junit4]   2> 3025358 WARN
> (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ]
> o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
>    [junit4]   2> 3025360 INFO
> (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ]
> o.m.log jetty-6.1.26
>    [junit4]   2> 3025373 INFO
> (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ]
> o.m.log Extract
> jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode
> to ./temp/Jetty_localhost_51889_datanode____.q33tks/webapp
>    [junit4]   2> 3025406 INFO  (IPC Server handler 0 on 50235) [    ]
> BlockStateChange BLOCK* processReport: from storage
> DS-1fad80c6-90eb-46db-9c7d-86281455e07f node
> DatanodeRegistration(127.0.0.1,
> datanodeUuid=10f01f83-63a4-4333-9074-5abef760e258, infoPort=58802,
> ipcPort=54385, storageInfo=lv=-56;cid=testClusterID;nsid=1611814338;c=0),
> blocks: 0, hasStaleStorages: true, processing time: 0 msecs
>    [junit4]   2> 3025406 INFO  (IPC Server handler 0 on 50235) [    ]
> BlockStateChange BLOCK* processReport: from storage
> DS-40d84459-703f-40dc-8e56-8cc0330ada3e node
> DatanodeRegistration(127.0.0.1,
> datanodeUuid=10f01f83-63a4-4333-9074-5abef760e258, infoPort=58802,
> ipcPort=54385, storageInfo=lv=-56;cid=testClusterID;nsid=1611814338;c=0),
> blocks: 0, hasStaleStorages: false, processing time: 0 msecs
>    [junit4]   2> 3025499 INFO
> (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ]
> o.m.log NO JSP Support for /, did not find
> org.apache.jasper.servlet.JspServlet
>    [junit4]   2> 3025706 INFO
> (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ]
> o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost
> :51889
>    [junit4]   2> 3025803 INFO  (IPC Server handler 8 on 50235) [    ]
> BlockStateChange BLOCK* processReport: from storage
> DS-a361307c-b633-4137-b8a8-4e378c07432e node
> DatanodeRegistration(127.0.0.1,
> datanodeUuid=e4622669-05f4-4517-8b93-a00030a5b52a, infoPort=51889,
> ipcPort=57183, storageInfo=lv=-56;cid=testClusterID;nsid=1611814338;c=0),
> blocks: 0, hasStaleStorages: true, processing time: 1 msecs
>    [junit4]   2> 3025803 INFO  (IPC Server handler 8 on 50235) [    ]
> BlockStateChange BLOCK* processReport: from storage
> DS-11a4f7cc-a74a-49ea-be8b-584a47ab345d node
> DatanodeRegistration(127.0.0.1,
> datanodeUuid=e4622669-05f4-4517-8b93-a00030a5b52a, infoPort=51889,
> ipcPort=57183, storageInfo=lv=-56;cid=testClusterID;nsid=1611814338;c=0),
> blocks: 0, hasStaleStorages: false, processing time: 0 msecs
>    [junit4]   2> 3025887 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
>    [junit4]   2> 3025888 INFO  (Thread-6976) [    ] o.a.s.c.ZkTestServer
> client port:0.0.0.0/0.0.0.0:0
>    [junit4]   2> 3025888 INFO  (Thread-6976) [    ] o.a.s.c.ZkTestServer
> Starting server
>    [junit4]   2> 3025988 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.ZkTestServer start zk server on port:59312
>    [junit4]   2> 3025988 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 3025989 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 3025992 INFO  (zkCallback-2011-thread-1) [    ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@36a71322
> name:ZooKeeperConnection Watcher:127.0.0.1:59312 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 3025992 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 3025992 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 3025992 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.SolrZkClient makePath: /solr
>    [junit4]   2> 3025995 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 3025995 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 3025996 INFO  (zkCallback-2012-thread-1) [    ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@629fafa0
> name:ZooKeeperConnection Watcher:127.0.0.1:59312/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 3025996 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 3025997 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 3025997 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.SolrZkClient makePath: /collections/collection1
>    [junit4]   2> 3025998 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
>    [junit4]   2> 3026000 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
>    [junit4]   2> 3026001 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
>    [junit4]   2> 3026002 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.AbstractZkTestCase put
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml
> to /configs/conf1/solrconfig.xml
>    [junit4]   2> 3026002 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
>    [junit4]   2> 3026005 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.AbstractZkTestCase put
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml
> to /configs/conf1/schema.xml
>    [junit4]   2> 3026005 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
>    [junit4]   2> 3026006 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.AbstractZkTestCase put
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
> to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
>    [junit4]   2> 3026007 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.SolrZkClient makePath:
> /configs/conf1/solrconfig.snippet.randomindexconfig.xml
>    [junit4]   2> 3026008 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.AbstractZkTestCase put
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
> to /configs/conf1/stopwords.txt
>    [junit4]   2> 3026008 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
>    [junit4]   2> 3026010 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.AbstractZkTestCase put
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt
> to /configs/conf1/protwords.txt
>    [junit4]   2> 3026010 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
>    [junit4]   2> 3026011 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.AbstractZkTestCase put
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml
> to /configs/conf1/currency.xml
>    [junit4]   2> 3026012 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
>    [junit4]   2> 3026013 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.AbstractZkTestCase put
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
> to /configs/conf1/enumsConfig.xml
>    [junit4]   2> 3026013 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
>    [junit4]   2> 3026015 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.AbstractZkTestCase put
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
> to /configs/conf1/open-exchange-rates.json
>    [junit4]   2> 3026015 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
>    [junit4]   2> 3026017 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.AbstractZkTestCase put
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
> to /configs/conf1/mapping-ISOLatin1Accent.txt
>    [junit4]   2> 3026017 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
>    [junit4]   2> 3026018 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.AbstractZkTestCase put
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
> to /configs/conf1/old_synonyms.txt
>    [junit4]   2> 3026018 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
>    [junit4]   2> 3026020 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.AbstractZkTestCase put
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
> to /configs/conf1/synonyms.txt
>    [junit4]   2> 3026020 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
>    [junit4]   2> 3026315 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.SolrTestCaseJ4 Writing core.properties file to
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/control-001/cores/collection1
>    [junit4]   2> 3026317 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.e.j.s.Server jetty-9.2.13.v20150730
>    [junit4]   2> 3026319 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4c048f9a
> {/rz/r,null,AVAILABLE}
>    [junit4]   2> 3026319 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.e.j.s.ServerConnector Started ServerConnector@20dfb26a{HTTP/1.1}{
> 127.0.0.1:49192}
>    [junit4]   2> 3026319 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.e.j.s.Server Started @3028954ms
>    [junit4]   2> 3026320 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.s.e.JettySolrRunner Jetty properties:
> {solr.data.dir=hdfs://localhost:50235/hdfs__localhost_50235__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001_tempDir-002_control_data,
> hostPort=49192,
> coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/control-001/cores,
> hostContext=/rz/r}
>    [junit4]   2> 3026320 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init():
> sun.misc.Launcher$AppClassLoader@3827de24
>    [junit4]   2> 3026320 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory:
> '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/control-001'
>    [junit4]   2> 3026320 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
>    [junit4]   2> 3026320 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find
> system property or JNDI)
>    [junit4]   2> 3026320 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 3026324 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 3026325 INFO  (zkCallback-2013-thread-1) [    ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@4b42d03
> name:ZooKeeperConnection Watcher:127.0.0.1:59312/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 3026325 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 3026325 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 3026326 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in
> ZooKeeper)
>    [junit4]   2> 3026326 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.SolrXmlConfig Loading container configuration from
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/control-001/solr.xml
>    [junit4]   2> 3026340 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.CorePropertiesLocator Config-defined core root directory:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/control-001/cores
>    [junit4]   2> 3026340 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.CoreContainer New CoreContainer 1615142391
>    [junit4]   2> 3026340 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.CoreContainer Loading cores into CoreContainer
> [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/control-001]
>    [junit4]   2> 3026340 WARN
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.CoreContainer Couldn't add files from
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/control-001/lib
> to classpath:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/control-001/lib
>    [junit4]   2> 3026340 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout :
> 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost :
> 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize :
> 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy :
> false,useRetries : false,
>    [junit4]   2> 3026342 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with
> params: socketTimeout=340000&connTimeout=45000&retry=true
>    [junit4]   2> 3026342 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
>    [junit4]   2> 3026342 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.l.LogWatcher Registering Log Listener [Log4j
> (org.slf4j.impl.Log4jLoggerFactory)]
>    [junit4]   2> 3026342 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59312/solr
>    [junit4]   2> 3026342 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.ZkController zkHost includes chroot
>    [junit4]   2> 3026342 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 3026343 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 3026344 INFO  (zkCallback-2015-thread-1) [    ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@6364e675
> name:ZooKeeperConnection Watcher:127.0.0.1:59312 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 3026344 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 3026345 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 3026346 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573])
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.ConnectionManager Waiting for
> client to connect to ZooKeeper
>    [junit4]   2> 3026347 INFO
> (zkCallback-2016-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@707b168b
> name:ZooKeeperConnection Watcher:127.0.0.1:59312/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 3026347 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573])
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.ConnectionManager Client is
> connected to ZooKeeper
>    [junit4]   2> 3026348 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573])
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.SolrZkClient makePath:
> /overseer/queue
>    [junit4]   2> 3026350 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573])
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.SolrZkClient makePath:
> /overseer/collection-queue-work
>    [junit4]   2> 3026353 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573])
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.SolrZkClient makePath:
> /overseer/collection-map-running
>    [junit4]   2> 3026355 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573])
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.SolrZkClient makePath:
> /overseer/collection-map-completed
>    [junit4]   2> 3026356 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573])
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.SolrZkClient makePath:
> /overseer/collection-map-failure
>    [junit4]   2> 3026358 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573])
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.SolrZkClient makePath:
> /live_nodes
>    [junit4]   2> 3026359 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573])
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.SolrZkClient makePath:
> /aliases.json
>    [junit4]   2> 3026361 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573])
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.SolrZkClient makePath:
> /clusterstate.json
>    [junit4]   2> 3026362 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573])
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.SolrZkClient makePath:
> /security.json
>    [junit4]   2> 3026362 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573])
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.ZkController Register node as live
> in ZooKeeper:/live_nodes/127.0.0.1:49192_rz%2Fr
>    [junit4]   2> 3026363 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573])
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.SolrZkClient makePath:
> /live_nodes/127.0.0.1:49192_rz%2Fr
>    [junit4]   2> 3026364 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573])
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.SolrZkClient makePath:
> /overseer_elect
>    [junit4]   2> 3026365 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573])
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.SolrZkClient makePath:
> /overseer_elect/election
>    [junit4]   2> 3026366 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573])
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.Overseer Overseer (id=null) closing
>    [junit4]   2> 3026367 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573])
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.LeaderElector Joined leadership
> election with path:
> /overseer_elect/election/95121056681558020-127.0.0.1:49192_rz%2Fr-n_0000000000
>    [junit4]   2> 3026367 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573])
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.OverseerElectionContext I am going
> to be the leader 127.0.0.1:49192_rz%2Fr
>    [junit4]   2> 3026367 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573])
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.SolrZkClient makePath:
> /overseer_elect/leader
>    [junit4]   2> 3026369 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573])
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.Overseer Overseer
> (id=95121056681558020-127.0.0.1:49192_rz%2Fr-n_0000000000) starting
>    [junit4]   2> 3026370 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573])
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.SolrZkClient makePath:
> /overseer/queue-work
>    [junit4]   2> 3026375 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573])
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.OverseerAutoReplicaFailoverThread
> Starting OverseerAutoReplicaFailoverThread
> autoReplicaFailoverWorkLoopDelay=10000
> autoReplicaFailoverWaitAfterExpiration=30000
> autoReplicaFailoverBadNodeExpiration=60000
>    [junit4]   2> 3026376 INFO
> (OverseerCollectionConfigSetProcessor-95121056681558020-127.0.0.1:49192_rz%2Fr-n_0000000000)
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.OverseerTaskProcessor Process
> current queue of overseer operations
>    [junit4]   2> 3026376 INFO
> (OverseerStateUpdate-95121056681558020-127.0.0.1:49192_rz%2Fr-n_0000000000)
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.Overseer Starting to work on the
> main queue
>    [junit4]   2> 3026376 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573])
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.ZkStateReader Updating cluster
> state from ZooKeeper...
>    [junit4]   2> 3026405 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573])
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.CoreContainer Security conf
> doesn't exist. Skipping setup for authorization module.
>    [junit4]   2> 3026405 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573])
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.CoreContainer No authentication
> plugin used.
>    [junit4]   2> 3026406 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573])
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.CorePropertiesLocator Looking for
> core definitions underneath
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/control-001/cores
>    [junit4]   2> 3026407 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573])
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.CoreDescriptor Created
> CoreDescriptor: {dataDir=data/, loadOnStartup=true,
> collection=control_collection, shard=, transient=false,
> config=solrconfig.xml, schema=schema.xml, name=collection1,
> configSetProperties=configsetprops.json, coreNodeName=}
>    [junit4]   2> 3026407 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573])
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.CorePropertiesLocator Found core
> collection1 in
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/control-001/cores/collection1
>    [junit4]   2> 3026407 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573])
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.CorePropertiesLocator Found 1 core
> definitions
>    [junit4]   2> 3026408 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573])
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.s.SolrDispatchFilter
> user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
>    [junit4]   2> 3026408 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1]
> o.a.s.c.ZkController publishing state=down
>    [junit4]   2> 3026408 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573])
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.s.SolrDispatchFilter
> SolrDispatchFilter.init() done
>    [junit4]   2> 3026408 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1]
> o.a.s.c.ZkController numShards not found on descriptor - reading it from
> system property
>    [junit4]   2> 3026410 INFO
> (OverseerStateUpdate-95121056681558020-127.0.0.1:49192_rz%2Fr-n_0000000000)
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.Overseer processMessage:
> queueSize: 1, message = {
>    [junit4]   2>   "shard":null,
>    [junit4]   2>   "node_name":"127.0.0.1:49192_rz%2Fr",
>    [junit4]   2>   "core":"collection1",
>    [junit4]   2>   "roles":null,
>    [junit4]   2>   "operation":"state",
>    [junit4]   2>   "base_url":"http://127.0.0.1:49192/rz/r",
>    [junit4]   2>   "numShards":"1",
>    [junit4]   2>   "collection":"control_collection",
>    [junit4]   2>   "state":"down"} current state version: 0
>    [junit4]   2> 3026410 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1]
> o.a.s.c.ZkController look for our core node name
>    [junit4]   2> 3026410 INFO
> (OverseerStateUpdate-95121056681558020-127.0.0.1:49192_rz%2Fr-n_0000000000)
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.o.ReplicaMutator Update state
> numShards=1 message={
>    [junit4]   2>   "shard":null,
>    [junit4]   2>   "node_name":"127.0.0.1:49192_rz%2Fr",
>    [junit4]   2>   "core":"collection1",
>    [junit4]   2>   "roles":null,
>    [junit4]   2>   "operation":"state",
>    [junit4]   2>   "base_url":"http://127.0.0.1:49192/rz/r",
>    [junit4]   2>   "numShards":"1",
>    [junit4]   2>   "collection":"control_collection",
>    [junit4]   2>   "state":"down"}
>    [junit4]   2> 3026411 INFO
> (OverseerStateUpdate-95121056681558020-127.0.0.1:49192_rz%2Fr-n_0000000000)
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.o.ClusterStateMutator building a
> new cName: control_collection
>    [junit4]   2> 3026411 INFO
> (OverseerStateUpdate-95121056681558020-127.0.0.1:49192_rz%2Fr-n_0000000000)
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.o.ReplicaMutator Assigning new
> node to shard shard=shard1
>    [junit4]   2> 3026412 INFO
> (zkCallback-2016-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.ZkStateReader A cluster state
> change: [WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
>    [junit4]   2> 3027411 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1]
> o.a.s.c.ZkController waiting to find shard id in clusterstate for
> collection1
>    [junit4]   2> 3027411 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1]
> o.a.s.c.ZkController Check for collection zkNode:control_collection
>    [junit4]   2> 3027411 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1]
> o.a.s.c.ZkController Collection zkNode exists
>    [junit4]   2> 3027411 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1]
> o.a.s.c.c.ZkStateReader Load collection config from:
> [/collections/control_collection]
>    [junit4]   2> 3027412 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1]
> o.a.s.c.c.ZkStateReader path=[/collections/control_collection]
> [configName]=[conf1] specified config exists in ZooKeeper
>    [junit4]   2> 3027412 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1]
> o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory:
> '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/control-001/cores/collection1'
>    [junit4]   2> 3027412 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1]
> o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
>    [junit4]   2> 3027412 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1]
> o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find
> system property or JNDI)
>    [junit4]   2> 3027414 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1]
> o.a.s.c.Config loaded config solrconfig.xml with version 0
>    [junit4]   2> 3027419 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1]
> o.a.s.c.SolrConfig current version of requestparams : -1
>    [junit4]   2> 3027431 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1]
> o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
>    [junit4]   2> 3027457 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1]
> o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
>    [junit4]   2> 3027464 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1]
> o.a.s.s.IndexSchema [collection1] Schema name=test
>    [junit4]   2> 3027658 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1]
> o.a.s.s.IndexSchema default search field in schema is text
>    [junit4]   2> 3027660 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1]
> o.a.s.s.IndexSchema unique key field: id
>    [junit4]   2> 3027661 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1]
> o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file
> currency.xml
>    [junit4]   2> 3027664 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1]
> o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file
> currency.xml
>    [junit4]   2> 3027678 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1]
> o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming
> default properties: Can't find resource 'configsetprops.json' in classpath
> or '/configs/conf1',
> cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
>    [junit4]   2> 3027679 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1]
> o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration
> from collection control_collection
>    [junit4]   2> 3027679 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
>    [junit4]   2> 3027679 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.HdfsDirectoryFactory
> solr.hdfs.home=hdfs://localhost:50235/solr_hdfs_home
>    [junit4]   2> 3027679 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication
> disabled
>    [junit4]   2> 3027679 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at
> [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/control-001/cores/collection1],
> dataDir=[hdfs://localhost:50235/solr_hdfs_home/control_collection/core_node1/data/]
>    [junit4]   2> 3027679 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding
> Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@11c0600a
>    [junit4]   2> 3027680 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for
> path
> hdfs://localhost:50235/solr_hdfs_home/control_collection/core_node1/data
>    [junit4]   2> 3027698 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for
> hdfs://localhost:50235/solr_hdfs_home/control_collection/core_node1/data
>    [junit4]   2> 3027699 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.SolrCore New index directory detected: old=null
> new=hdfs://localhost:50235/solr_hdfs_home/control_collection/core_node1/data/index/
>    [junit4]   2> 3027709 WARN
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory
> 'hdfs:/localhost:50235/solr_hdfs_home/control_collection/core_node1/data/index'
> doesn't exist. Creating new index...
>    [junit4]   2> 3027710 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for
> path
> hdfs://localhost:50235/solr_hdfs_home/control_collection/core_node1/data/index
>    [junit4]   2> 3027718 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache
> [1] with direct memory allocation set to [true]
>    [junit4]   2> 3027718 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory
> usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608]
> bytes
>    [junit4]   2> 3027718 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new global HDFS
> BlockCache
>    [junit4]   2> 3027728 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
>    [junit4]   2> 3027728 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for
> hdfs://localhost:50235/solr_hdfs_home/control_collection/core_node1/data/index
>    [junit4]   2> 3027729 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class
> org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy:
> minMergeSize=1677721, mergeFactor=35, maxMergeSize=2147483648,
> maxMergeSizeForForcedMerge=9223372036854775807,
> calibrateSizeByDeletes=true, maxMergeDocs=2147483647,
> maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5070751795047811]
>    [junit4]   2> 3027748 INFO  (IPC Server handler 9 on 50235) [    ]
> BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:38880
> is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION,
> primaryNodeIndex=-1,
> replicas=[ReplicaUnderConstruction[[DISK]DS-40d84459-703f-40dc-8e56-8cc0330ada3e:NORMAL:127.0.0.1:46787|RBW],
> ReplicaUnderConstruction[[DISK]DS-a361307c-b633-4137-b8a8-4e378c07432e:NORMAL:127.0.0.1:38880|RBW]]}
> size 0
>    [junit4]   2> 3027750 INFO  (IPC Server handler 2 on 50235) [    ]
> BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:46787
> is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION,
> primaryNodeIndex=-1,
> replicas=[ReplicaUnderConstruction[[DISK]DS-40d84459-703f-40dc-8e56-8cc0330ada3e:NORMAL:127.0.0.1:46787|RBW],
> ReplicaUnderConstruction[[DISK]DS-a361307c-b633-4137-b8a8-4e378c07432e:NORMAL:127.0.0.1:38880|RBW]]}
> size 0
>    [junit4]   2> 3027753 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit:
> commits: num=1
>    [junit4]   2>        commit{dir=NRTCachingDirectory(BlockDirectory(
> HdfsDirectory@hdfs://localhost:50235/solr_hdfs_home/control_collection/core_node1/data/index
> lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@70755032);
> maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
>    [junit4]   2> 3027753 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
>    [junit4]   2> 3027755 INFO
> (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:49192_rz%2Fr
> c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore
> Looking for old index directories to cleanup for core collection1 in
> hdfs://localhost:50235/solr_hdfs_home/control_collection/core_node1/data/
>    [junit4]   2> 3027758 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating
> updateRequestProcessorChain "nodistrib"
>    [junit4]   2> 3027758 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating
> updateRequestProcessorChain "dedupe"
>    [junit4]   2> 3027758 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting
> DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
>    [junit4]   2> 3027758 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating
> updateRequestProcessorChain "stored_sig"
>    [junit4]   2> 3027758 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting
> DistributedUpdateProcessorFactory into updateRequestProcessorChain
> "stored_sig"
>    [junit4]   2> 3027758 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating
> updateRequestProcessorChain "distrib-dup-test-chain-explicit"
>    [junit4]   2> 3027759 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating
> updateRequestProcessorChain "distrib-dup-test-chain-implicit"
>    [junit4]   2> 3027759 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting
> DistributedUpdateProcessorFactory into updateRequestProcessorChain
> "distrib-dup-test-chain-implicit"
>    [junit4]   2> 3027759 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as
> default, creating implicit default
>    [junit4]   2> 3027759 WARN
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler
> {type = requestHandler,name = /dump,class = DumpRequestHandler,args =
> {defaults={a=A,b=B}}}
>    [junit4]   2> 3027762 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
>    [junit4]   2> 3027763 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
>    [junit4]   2> 3027764 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
>    [junit4]   2> 3027765 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
>    [junit4]   2> 3027768 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.RequestHandlers Registered paths:
> /update/json/docs,/admin/properties,/get,/update/json,/schema,/dump,/update,/admin/plugins,/admin/mbeans,/admin/system,/admin/logging,/admin/ping,/replication,/admin/file,/update/csv,standard,/admin/segments,/admin/luke,/admin/threads,/config
>    [junit4]   2> 3027770 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.SolrCore Using default statsCache cache:
> org.apache.solr.search.stats.LocalStatsCache
>    [junit4]   2> 3027770 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation:
> org.apache.solr.update.HdfsUpdateLog
>    [junit4]   2> 3027770 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: dataDir=
> defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10
> tlogDfsReplication=2
>    [junit4]   2> 3027782 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for
> 15000ms;
>    [junit4]   2> 3027782 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
>    [junit4]   2> 3027784 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class
> org.apache.lucene.index.MockRandomMergePolicy:
> org.apache.lucene.index.MockRandomMergePolicy@7835e1fd
>    [junit4]   2> 3027793 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit:
> commits: num=1
>    [junit4]   2>        commit{dir=NRTCachingDirectory(BlockDirectory(
> HdfsDirectory@hdfs://localhost:50235/solr_hdfs_home/control_collection/core_node1/data/index
> lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@70755032);
> maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
>    [junit4]   2> 3027793 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
>    [junit4]   2> 3027793 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@197e4ac8[collection1]
> main
>    [junit4]   2> 3027793 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:
> [/collections/control_collection]
>    [junit4]   2> 3027794 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.c.ZkStateReader
> path=[/collections/control_collection] [configName]=[conf1] specified
> config exists in ZooKeeper
>    [junit4]   2> 3027794 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based
> storage for the RestManager with znodeBase: /configs/conf1
>    [junit4]   2> 3027794 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO
> with znodeBase: /configs/conf1
>    [junit4]   2> 3027794 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs:
> {}
>    [junit4]   2> 3027795 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json
> using ZooKeeperStorageIO:path=/configs/conf1
>    [junit4]   2> 3027795 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode
> /configs/conf1/_rest_managed.json
>    [junit4]   2> 3027795 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path
> _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
>    [junit4]   2> 3027795 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.r.RestManager Initializing 0 registered
> ManagedResources
>    [junit4]   2> 3027795 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for
> 10000
>    [junit4]   2> 3027797 INFO
> (searcherExecutor-6607-thread-1-processing-s:shard1 n:127.0.0.1:49192_rz%2Fr
> x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:49192_rz%2Fr
> c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore
> [collection1] Registered new searcher Searcher@197e4ac8[collection1]
> main{ExitableDirectoryReader(UninvertingDirectoryReader())}
>    [junit4]   2> 3027798 WARN
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.s.h.HdfsLocalityReporter Could not retrieve locality
> information for hdfs://localhost:36278/solr3 due to exception:
> java.net.ConnectException: Call From lucene1-us-west/10.41.0.5 to
> localhost:36278 failed on connection exception: java.net.ConnectException:
> Connection refused; For more details see:
> http://wiki.apache.org/hadoop/ConnectionRefused
>    [junit4]   2> 3027798 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to
> seed version buckets
>    [junit4]   2> 3027798 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_
> for 256 version buckets from index
>    [junit4]   2> 3027798 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot
> seed version bucket highest value from index
>    [junit4]   2> 3027798 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.UpdateLog Could not find max version in index or
> recent updates, using new clock 1521936908906659840
>    [junit4]   2> 3027798 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with
> highest version 1521936908906659840
>    [junit4]   2> 3027798 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
>    [junit4]   2> 3027799 INFO
> (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1
> x:collection1] o.a.s.c.CoreContainer registering core: collection1
>    [junit4]   2> 3027799 INFO
> (coreZkRegister-6600-thread-1-processing-s:shard1 n:127.0.0.1:49192_rz%2Fr
> x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:49192_rz%2Fr
> c:control_collection s:shard1 r:core_node1 x:collection1]
> o.a.s.c.ZkController Register replica - core:collection1 address:
> http://127.0.0.1:49192/rz/r collection:control_collection shard:shard1
>    [junit4]   2> 3027800 INFO
> (coreZkRegister-6600-thread-1-processing-s:shard1 n:127.0.0.1:49192_rz%2Fr
> x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:49192_rz%2Fr
> c:control_collection s:shard1 r:core_node1 x:collection1]
> o.a.s.c.c.SolrZkClient makePath:
> /collections/control_collection/leader_elect/shard1/election
>    [junit4]   2> 3027803 INFO
> (coreZkRegister-6600-thread-1-processing-s:shard1 n:127.0.0.1:49192_rz%2Fr
> x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:49192_rz%2Fr
> c:control_collection s:shard1 r:core_node1 x:collection1]
> o.a.s.c.LeaderElector Joined leadership election with path:
> /collections/control_collection/leader_elect/shard1/election/95121056681558020-core_node1-n_0000000000
>    [junit4]   2> 3027804 INFO
> (coreZkRegister-6600-thread-1-processing-s:shard1 n:127.0.0.1:49192_rz%2Fr
> x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:49192_rz%2Fr
> c:control_collection s:shard1 r:core_node1 x:collection1]
> o.a.s.c.ShardLeaderElectionContext Running the leader process for
> shard=shard1 and weAreReplacement=false and leaderVoteWait=10000
>    [junit4]   2> 3027805 INFO
> (coreZkRegister-6600-thread-1-processing-s:shard1 n:127.0.0.1:49192_rz%2Fr
> x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:49192_rz%2Fr
> c:control_collection s:shard1 r:core_node1 x:collection1]
> o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
>    [junit4]   2> 3027805 INFO
> (coreZkRegister-6600-thread-1-processing-s:shard1 n:127.0.0.1:49192_rz%2Fr
> x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:49192_rz%2Fr
> c:control_collection s:shard1 r:core_node1 x:collection1]
> o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
>    [junit4]   2> 3027805 INFO
> (coreZkRegister-6600-thread-1-processing-s:shard1 n:127.0.0.1:49192_rz%2Fr
> x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:49192_rz%2Fr
> c:control_collection s:shard1 r:core_node1 x:collection1]
> o.a.s.c.SyncStrategy Sync replicas to
> http://127.0.0.1:49192/rz/r/collection1/
>    [junit4]   2> 3027805 INFO
> (coreZkRegister-6600-thread-1-processing-s:shard1 n:127.0.0.1:49192_rz%2Fr
> x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:49192_rz%2Fr
> c:control_collection s:shard1 r:core_node1 x:collection1]
> o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
>    [junit4]   2> 3027805 INFO
> (coreZkRegister-6600-thread-1-processing-s:shard1 n:127.0.0.1:49192_rz%2Fr
> x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:49192_rz%2Fr
> c:control_collection s:shard1 r:core_node1 x:collection1]
> o.a.s.c.SyncStrategy http://127.0.0.1:49192/rz/r/collection1/ has no
> replicas
>    [junit4]   2> 3027805 INFO
> (OverseerStateUpdate-95121056681558020-127.0.0.1:49192_rz%2Fr-n_0000000000)
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.Overseer processMessage:
> queueSize: 1, message = {
>    [junit4]   2>   "operation":"leader",
>    [junit4]   2>   "shard":"shard1",
>    [junit4]   2>   "collection":"control_collection"} current state
> version: 1
>    [junit4]   2> 3027806 INFO
> (coreZkRegister-6600-thread-1-processing-s:shard1 n:127.0.0.1:49192_rz%2Fr
> x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:49192_rz%2Fr
> c:control_collection s:shard1 r:core_node1 x:collection1]
> o.a.s.c.c.SolrZkClient makePath:
> /collections/control_collection/leaders/shard1
>    [junit4]   2> 3027808 INFO
> (coreZkRegister-6600-thread-1-processing-s:shard1 n:127.0.0.1:49192_rz%2Fr
> x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:49192_rz%2Fr
> c:control_collection s:shard1 r:core_node1 x:collection1]
> o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
>    [junit4]   2> 3027810 INFO
> (coreZkRegister-6600-thread-1-processing-s:shard1 n:127.0.0.1:49192_rz%2Fr
> x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:49192_rz%2Fr
> c:control_collection s:shard1 r:core_node1 x:collection1]
> o.a.s.c.ShardLeaderElectionContext I am the new leader:
> http://127.0.0.1:49192/rz/r/collection1/ shard1
>    [junit4]   2> 3027810 INFO
> (OverseerStateUpdate-95121056681558020-127.0.0.1:49192_rz%2Fr-n_0000000000)
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.Overseer processMessage:
> queueSize: 1, message = {
>    [junit4]   2>   "operation":"leader",
>    [junit4]   2>   "shard":"shard1",
>    [junit4]   2>   "collection":"control_collection",
>    [junit4]   2>   "base_url":"http://127.0.0.1:49192/rz/r",
>    [junit4]   2>   "core":"collection1",
>    [junit4]   2>   "state":"active"} current state version: 1
>    [junit4]   2> 3027910 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 3027911 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 3027912 INFO
> (zkCallback-2016-thread-1-processing-n:127.0.0.1:49192_rz%2Fr)
> [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.ZkStateReader A cluster state
> change: [WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
>    [junit4]   2> 3027913 INFO  (zkCallback-2018-thread-1) [    ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@6a282588
> name:ZooKeeperConnection Watcher:127.0.0.1:59312/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 3027913 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 3027913 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 3027913 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
>    [junit4]   2> 3027915 INFO
> (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ]
> o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection
> loss:false
>    [junit4]   2> 3027961 INFO
> (coreZkRegister-6600-thread-1-processing-s:shard1 n:127.0.0.1:49192_rz%2Fr
> x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:49192_rz%2Fr
> c:

-- 
- Mark
about.me/markrmiller

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1061 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1061/

3 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest.test

Error Message:
The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!

Stack Trace:
java.lang.AssertionError: The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
	at __randomizedtesting.SeedInfo.seed([EA3B02E14E63D573:626F3D3BE09FB88B]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.ChaosMonkey.stopTheMonkey(ChaosMonkey.java:560)
	at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test(ChaosMonkeySafeLeaderTest.java:146)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:965)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:940)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest

Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest:     1) Thread[id=15765, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeySafeLeaderTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:136)    2) Thread[id=15764, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeySafeLeaderTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:136)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest: 
   1) Thread[id=15765, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:136)
   2) Thread[id=15764, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:136)
	at __randomizedtesting.SeedInfo.seed([EA3B02E14E63D573]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=15764, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-HdfsChaosMonkeySafeLeaderTest]         at java.lang.Integer.getChars(Integer.java:372)         at java.lang.Integer.toString(Integer.java:332)         at java.lang.Integer.toString(Integer.java:739)         at org.apache.solr.client.solrj.util.ClientUtils.writeVal(ClientUtils.java:164)         at org.apache.solr.client.solrj.util.ClientUtils.writeXML(ClientUtils.java:132)         at org.apache.solr.client.solrj.request.UpdateRequest.writeXML(UpdateRequest.java:413)         at org.apache.solr.client.solrj.request.UpdateRequest.getXML(UpdateRequest.java:329)         at org.apache.solr.client.solrj.request.RequestWriter.getContentStream(RequestWriter.java:67)         at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:95)         at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:105)         at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:356)         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:239)         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)         at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)         at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:167)         at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:177)         at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:116)    2) Thread[id=15765, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-HdfsChaosMonkeySafeLeaderTest]         at java.lang.Integer.getChars(Integer.java:372)         at java.lang.Integer.toString(Integer.java:332)         at java.lang.Integer.toString(Integer.java:739)         at org.apache.solr.client.solrj.util.ClientUtils.writeVal(ClientUtils.java:164)         at org.apache.solr.client.solrj.util.ClientUtils.writeXML(ClientUtils.java:132)         at org.apache.solr.client.solrj.request.UpdateRequest.writeXML(UpdateRequest.java:413)         at org.apache.solr.client.solrj.request.UpdateRequest.getXML(UpdateRequest.java:329)         at org.apache.solr.client.solrj.request.RequestWriter.getContentStream(RequestWriter.java:67)         at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:95)         at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:105)         at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:356)         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:239)         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)         at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)         at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:167)         at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:177)         at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:116)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=15764, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
        at java.lang.Integer.getChars(Integer.java:372)
        at java.lang.Integer.toString(Integer.java:332)
        at java.lang.Integer.toString(Integer.java:739)
        at org.apache.solr.client.solrj.util.ClientUtils.writeVal(ClientUtils.java:164)
        at org.apache.solr.client.solrj.util.ClientUtils.writeXML(ClientUtils.java:132)
        at org.apache.solr.client.solrj.request.UpdateRequest.writeXML(UpdateRequest.java:413)
        at org.apache.solr.client.solrj.request.UpdateRequest.getXML(UpdateRequest.java:329)
        at org.apache.solr.client.solrj.request.RequestWriter.getContentStream(RequestWriter.java:67)
        at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:95)
        at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:105)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:356)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:239)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:167)
        at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:177)
        at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:116)
   2) Thread[id=15765, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
        at java.lang.Integer.getChars(Integer.java:372)
        at java.lang.Integer.toString(Integer.java:332)
        at java.lang.Integer.toString(Integer.java:739)
        at org.apache.solr.client.solrj.util.ClientUtils.writeVal(ClientUtils.java:164)
        at org.apache.solr.client.solrj.util.ClientUtils.writeXML(ClientUtils.java:132)
        at org.apache.solr.client.solrj.request.UpdateRequest.writeXML(UpdateRequest.java:413)
        at org.apache.solr.client.solrj.request.UpdateRequest.getXML(UpdateRequest.java:329)
        at org.apache.solr.client.solrj.request.RequestWriter.getContentStream(RequestWriter.java:67)
        at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:95)
        at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:105)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:356)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:239)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:167)
        at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:177)
        at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:116)
	at __randomizedtesting.SeedInfo.seed([EA3B02E14E63D573]:0)




Build Log:
[...truncated 11542 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/init-core-data-001
   [junit4]   2> 3024380 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 3024380 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /rz/r
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 3024425 WARN  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 3024434 WARN  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 3024437 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 3024450 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_40848_hdfs____.j54i3a/webapp
   [junit4]   2> 3024574 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 3024791 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40848
   [junit4]   2> 3024870 WARN  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 3024871 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 3024886 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_58802_datanode____fny2am/webapp
   [junit4]   2> 3025001 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 3025322 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:58802
   [junit4]   2> 3025358 WARN  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 3025360 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 3025373 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_51889_datanode____.q33tks/webapp
   [junit4]   2> 3025406 INFO  (IPC Server handler 0 on 50235) [    ] BlockStateChange BLOCK* processReport: from storage DS-1fad80c6-90eb-46db-9c7d-86281455e07f node DatanodeRegistration(127.0.0.1, datanodeUuid=10f01f83-63a4-4333-9074-5abef760e258, infoPort=58802, ipcPort=54385, storageInfo=lv=-56;cid=testClusterID;nsid=1611814338;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 3025406 INFO  (IPC Server handler 0 on 50235) [    ] BlockStateChange BLOCK* processReport: from storage DS-40d84459-703f-40dc-8e56-8cc0330ada3e node DatanodeRegistration(127.0.0.1, datanodeUuid=10f01f83-63a4-4333-9074-5abef760e258, infoPort=58802, ipcPort=54385, storageInfo=lv=-56;cid=testClusterID;nsid=1611814338;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 3025499 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 3025706 INFO  (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[EA3B02E14E63D573]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:51889
   [junit4]   2> 3025803 INFO  (IPC Server handler 8 on 50235) [    ] BlockStateChange BLOCK* processReport: from storage DS-a361307c-b633-4137-b8a8-4e378c07432e node DatanodeRegistration(127.0.0.1, datanodeUuid=e4622669-05f4-4517-8b93-a00030a5b52a, infoPort=51889, ipcPort=57183, storageInfo=lv=-56;cid=testClusterID;nsid=1611814338;c=0), blocks: 0, hasStaleStorages: true, processing time: 1 msecs
   [junit4]   2> 3025803 INFO  (IPC Server handler 8 on 50235) [    ] BlockStateChange BLOCK* processReport: from storage DS-11a4f7cc-a74a-49ea-be8b-584a47ab345d node DatanodeRegistration(127.0.0.1, datanodeUuid=e4622669-05f4-4517-8b93-a00030a5b52a, infoPort=51889, ipcPort=57183, storageInfo=lv=-56;cid=testClusterID;nsid=1611814338;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 3025887 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 3025888 INFO  (Thread-6976) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 3025888 INFO  (Thread-6976) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 3025988 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.ZkTestServer start zk server on port:59312
   [junit4]   2> 3025988 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3025989 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3025992 INFO  (zkCallback-2011-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@36a71322 name:ZooKeeperConnection Watcher:127.0.0.1:59312 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3025992 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3025992 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3025992 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 3025995 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3025995 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3025996 INFO  (zkCallback-2012-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@629fafa0 name:ZooKeeperConnection Watcher:127.0.0.1:59312/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3025996 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3025997 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3025997 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 3025998 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 3026000 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 3026001 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 3026002 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 3026002 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 3026005 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 3026005 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 3026006 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3026007 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3026008 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 3026008 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 3026010 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 3026010 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 3026011 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 3026012 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 3026013 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 3026013 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 3026015 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3026015 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3026017 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3026017 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3026018 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 3026018 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 3026020 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 3026020 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 3026315 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/control-001/cores/collection1
   [junit4]   2> 3026317 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 3026319 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4c048f9a{/rz/r,null,AVAILABLE}
   [junit4]   2> 3026319 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.e.j.s.ServerConnector Started ServerConnector@20dfb26a{HTTP/1.1}{127.0.0.1:49192}
   [junit4]   2> 3026319 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.e.j.s.Server Started @3028954ms
   [junit4]   2> 3026320 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:50235/hdfs__localhost_50235__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001_tempDir-002_control_data, hostPort=49192, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/control-001/cores, hostContext=/rz/r}
   [junit4]   2> 3026320 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 3026320 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/control-001'
   [junit4]   2> 3026320 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3026320 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3026320 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3026324 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3026325 INFO  (zkCallback-2013-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4b42d03 name:ZooKeeperConnection Watcher:127.0.0.1:59312/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3026325 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3026325 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3026326 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 3026326 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/control-001/solr.xml
   [junit4]   2> 3026340 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/control-001/cores
   [junit4]   2> 3026340 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.CoreContainer New CoreContainer 1615142391
   [junit4]   2> 3026340 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/control-001]
   [junit4]   2> 3026340 WARN  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/control-001/lib
   [junit4]   2> 3026340 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 3026342 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 3026342 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3026342 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3026342 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59312/solr
   [junit4]   2> 3026342 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 3026342 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3026343 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3026344 INFO  (zkCallback-2015-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6364e675 name:ZooKeeperConnection Watcher:127.0.0.1:59312 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3026344 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3026345 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3026346 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3026347 INFO  (zkCallback-2016-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@707b168b name:ZooKeeperConnection Watcher:127.0.0.1:59312/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3026347 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3026348 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 3026350 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 3026353 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 3026355 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 3026356 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 3026358 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 3026359 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 3026361 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 3026362 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 3026362 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49192_rz%2Fr
   [junit4]   2> 3026363 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:49192_rz%2Fr
   [junit4]   2> 3026364 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 3026365 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 3026366 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3026367 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95121056681558020-127.0.0.1:49192_rz%2Fr-n_0000000000
   [junit4]   2> 3026367 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:49192_rz%2Fr
   [junit4]   2> 3026367 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 3026369 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.Overseer Overseer (id=95121056681558020-127.0.0.1:49192_rz%2Fr-n_0000000000) starting
   [junit4]   2> 3026370 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 3026375 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 3026376 INFO  (OverseerCollectionConfigSetProcessor-95121056681558020-127.0.0.1:49192_rz%2Fr-n_0000000000) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 3026376 INFO  (OverseerStateUpdate-95121056681558020-127.0.0.1:49192_rz%2Fr-n_0000000000) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 3026376 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3026405 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 3026405 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 3026406 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/control-001/cores
   [junit4]   2> 3026407 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {dataDir=data/, loadOnStartup=true, collection=control_collection, shard=, transient=false, config=solrconfig.xml, schema=schema.xml, name=collection1, configSetProperties=configsetprops.json, coreNodeName=}
   [junit4]   2> 3026407 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/control-001/cores/collection1
   [junit4]   2> 3026407 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 3026408 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 3026408 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 3026408 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 3026408 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3026410 INFO  (OverseerStateUpdate-95121056681558020-127.0.0.1:49192_rz%2Fr-n_0000000000) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:49192_rz%2Fr",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:49192/rz/r",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "state":"down"} current state version: 0
   [junit4]   2> 3026410 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 3026410 INFO  (OverseerStateUpdate-95121056681558020-127.0.0.1:49192_rz%2Fr-n_0000000000) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:49192_rz%2Fr",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:49192/rz/r",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "state":"down"}
   [junit4]   2> 3026411 INFO  (OverseerStateUpdate-95121056681558020-127.0.0.1:49192_rz%2Fr-n_0000000000) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 3026411 INFO  (OverseerStateUpdate-95121056681558020-127.0.0.1:49192_rz%2Fr-n_0000000000) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 3026412 INFO  (zkCallback-2016-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 3027411 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 3027411 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 3027411 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 3027411 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 3027412 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 3027412 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/control-001/cores/collection1'
   [junit4]   2> 3027412 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3027412 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3027414 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 3027419 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 3027431 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 3027457 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 3027464 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 3027658 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 3027660 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 3027661 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 3027664 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 3027678 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 3027679 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 3027679 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 3027679 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:50235/solr_hdfs_home
   [junit4]   2> 3027679 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 3027679 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/control-001/cores/collection1], dataDir=[hdfs://localhost:50235/solr_hdfs_home/control_collection/core_node1/data/]
   [junit4]   2> 3027679 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@11c0600a
   [junit4]   2> 3027680 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:50235/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 3027698 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:50235/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 3027699 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:50235/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 3027709 WARN  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:50235/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 3027710 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:50235/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 3027718 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 3027718 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 3027718 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new global HDFS BlockCache
   [junit4]   2> 3027728 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 3027728 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:50235/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 3027729 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=35, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5070751795047811]
   [junit4]   2> 3027748 INFO  (IPC Server handler 9 on 50235) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:38880 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-40d84459-703f-40dc-8e56-8cc0330ada3e:NORMAL:127.0.0.1:46787|RBW], ReplicaUnderConstruction[[DISK]DS-a361307c-b633-4137-b8a8-4e378c07432e:NORMAL:127.0.0.1:38880|RBW]]} size 0
   [junit4]   2> 3027750 INFO  (IPC Server handler 2 on 50235) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:46787 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-40d84459-703f-40dc-8e56-8cc0330ada3e:NORMAL:127.0.0.1:46787|RBW], ReplicaUnderConstruction[[DISK]DS-a361307c-b633-4137-b8a8-4e378c07432e:NORMAL:127.0.0.1:38880|RBW]]} size 0
   [junit4]   2> 3027753 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@hdfs://localhost:50235/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@70755032); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 3027753 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 3027755 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:50235/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 3027758 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 3027758 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 3027758 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 3027758 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3027758 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3027758 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 3027759 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3027759 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3027759 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 3027759 WARN  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 3027762 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3027763 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3027764 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3027765 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3027768 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /update/json/docs,/admin/properties,/get,/update/json,/schema,/dump,/update,/admin/plugins,/admin/mbeans,/admin/system,/admin/logging,/admin/ping,/replication,/admin/file,/update/csv,standard,/admin/segments,/admin/luke,/admin/threads,/config
   [junit4]   2> 3027770 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 3027770 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 3027770 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 3027782 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms; 
   [junit4]   2> 3027782 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 3027784 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@7835e1fd
   [junit4]   2> 3027793 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@hdfs://localhost:50235/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@70755032); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 3027793 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 3027793 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@197e4ac8[collection1] main
   [junit4]   2> 3027793 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 3027794 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 3027794 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 3027794 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 3027794 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 3027795 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3027795 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 3027795 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3027795 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 3027795 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 3027797 INFO  (searcherExecutor-6607-thread-1-processing-s:shard1 n:127.0.0.1:49192_rz%2Fr x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@197e4ac8[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 3027798 WARN  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Could not retrieve locality information for hdfs://localhost:36278/solr3 due to exception: java.net.ConnectException: Call From lucene1-us-west/10.41.0.5 to localhost:36278 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
   [junit4]   2> 3027798 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 3027798 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 3027798 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 3027798 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1521936908906659840
   [junit4]   2> 3027798 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 1521936908906659840
   [junit4]   2> 3027798 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 3027799 INFO  (coreLoadExecutor-6606-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 3027799 INFO  (coreZkRegister-6600-thread-1-processing-s:shard1 n:127.0.0.1:49192_rz%2Fr x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:49192/rz/r collection:control_collection shard:shard1
   [junit4]   2> 3027800 INFO  (coreZkRegister-6600-thread-1-processing-s:shard1 n:127.0.0.1:49192_rz%2Fr x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 3027803 INFO  (coreZkRegister-6600-thread-1-processing-s:shard1 n:127.0.0.1:49192_rz%2Fr x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95121056681558020-core_node1-n_0000000000
   [junit4]   2> 3027804 INFO  (coreZkRegister-6600-thread-1-processing-s:shard1 n:127.0.0.1:49192_rz%2Fr x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 3027805 INFO  (coreZkRegister-6600-thread-1-processing-s:shard1 n:127.0.0.1:49192_rz%2Fr x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 3027805 INFO  (coreZkRegister-6600-thread-1-processing-s:shard1 n:127.0.0.1:49192_rz%2Fr x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 3027805 INFO  (coreZkRegister-6600-thread-1-processing-s:shard1 n:127.0.0.1:49192_rz%2Fr x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:49192/rz/r/collection1/
   [junit4]   2> 3027805 INFO  (coreZkRegister-6600-thread-1-processing-s:shard1 n:127.0.0.1:49192_rz%2Fr x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 3027805 INFO  (coreZkRegister-6600-thread-1-processing-s:shard1 n:127.0.0.1:49192_rz%2Fr x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:49192/rz/r/collection1/ has no replicas
   [junit4]   2> 3027805 INFO  (OverseerStateUpdate-95121056681558020-127.0.0.1:49192_rz%2Fr-n_0000000000) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 3027806 INFO  (coreZkRegister-6600-thread-1-processing-s:shard1 n:127.0.0.1:49192_rz%2Fr x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 3027808 INFO  (coreZkRegister-6600-thread-1-processing-s:shard1 n:127.0.0.1:49192_rz%2Fr x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 3027810 INFO  (coreZkRegister-6600-thread-1-processing-s:shard1 n:127.0.0.1:49192_rz%2Fr x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:49192/rz/r/collection1/ shard1
   [junit4]   2> 3027810 INFO  (OverseerStateUpdate-95121056681558020-127.0.0.1:49192_rz%2Fr-n_0000000000) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:49192/rz/r",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 3027910 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3027911 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3027912 INFO  (zkCallback-2016-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 3027913 INFO  (zkCallback-2018-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6a282588 name:ZooKeeperConnection Watcher:127.0.0.1:59312/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3027913 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3027913 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3027913 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3027915 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 3027961 INFO  (coreZkRegister-6600-thread-1-processing-s:shard1 n:127.0.0.1:49192_rz%2Fr x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:49192/rz/r/collection1/ and leader is http://127.0.0.1:49192/rz/r/collection1/
   [junit4]   2> 3027961 INFO  (coreZkRegister-6600-thread-1-processing-s:shard1 n:127.0.0.1:49192_rz%2Fr x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49192/rz/r
   [junit4]   2> 3027961 INFO  (coreZkRegister-6600-thread-1-processing-s:shard1 n:127.0.0.1:49192_rz%2Fr x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 3027961 INFO  (coreZkRegister-6600-thread-1-processing-s:shard1 n:127.0.0.1:49192_rz%2Fr x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 3027961 INFO  (coreZkRegister-6600-thread-1-processing-s:shard1 n:127.0.0.1:49192_rz%2Fr x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:49192_rz%2Fr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3027963 INFO  (OverseerStateUpdate-95121056681558020-127.0.0.1:49192_rz%2Fr-n_0000000000) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "node_name":"127.0.0.1:49192_rz%2Fr",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:49192/rz/r",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 3027964 INFO  (OverseerStateUpdate-95121056681558020-127.0.0.1:49192_rz%2Fr-n_0000000000) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "node_name":"127.0.0.1:49192_rz%2Fr",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:49192/rz/r",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3028066 INFO  (zkCallback-2016-thread-1-processing-n:127.0.0.1:49192_rz%2Fr) [n:127.0.0.1:49192_rz%2Fr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 3028066 INFO  (zkCallback-2018-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 3028200 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/shard-1-001/cores/collection1
   [junit4]   2> 3028201 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/shard-1-001
   [junit4]   2> 3028202 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 3028205 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5e3c3560{/rz/r,null,AVAILABLE}
   [junit4]   2> 3028205 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.e.j.s.ServerConnector Started ServerConnector@56b3e1a{HTTP/1.1}{127.0.0.1:48719}
   [junit4]   2> 3028205 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.e.j.s.Server Started @3030840ms
   [junit4]   2> 3028205 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:50235/hdfs__localhost_50235__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001_tempDir-002_jetty1, solrconfig=solrconfig.xml, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/shard-1-001/cores, hostPort=48719, hostContext=/rz/r}
   [junit4]   2> 3028206 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 3028206 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/shard-1-001'
   [junit4]   2> 3028206 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3028206 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3028206 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3028207 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3028208 INFO  (zkCallback-2019-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@21f506b0 name:ZooKeeperConnection Watcher:127.0.0.1:59312/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3028208 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3028209 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3028210 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 3028210 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/shard-1-001/solr.xml
   [junit4]   2> 3028223 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/shard-1-001/cores
   [junit4]   2> 3028223 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.CoreContainer New CoreContainer 1657783618
   [junit4]   2> 3028223 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/shard-1-001]
   [junit4]   2> 3028223 WARN  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_EA3B02E14E63D573-001/shard-1-001/lib
   [junit4]   2> 3028223 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 3028224 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 3028225 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3028225 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3028225 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59312/solr
   [junit4]   2> 3028225 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 3028225 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3028226 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3028227 INFO  (zkCallback-2021-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7b320ffc name:ZooKeeperConnection Watcher:127.0.0.1:59312 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3028227 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3028227 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3028229 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:48719_rz%2Fr    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3028230 INFO  (zkCallback-2022-thread-1-processing-n:127.0.0.1:48719_rz%2Fr) [n:127.0.0.1:48719_rz%2Fr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@487106d6 name:ZooKeeperConnection Watcher:127.0.0.1:59312/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3028230 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:48719_rz%2Fr    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3028234 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:48719_rz%2Fr    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3029240 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:48719_rz%2Fr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48719_rz%2Fr
   [junit4]   2> 3029240 INFO  (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[EA3B02E14E63D573]) [n:127.0.0.1:48719_rz%2Fr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:48719_rz%2Fr
   [junit4]   2> 3029241 INFO  (zkCallback-2022-thread-1-processing-n:127.0.0.1:48719_rz%2Fr) [n:127.0.0.1:48719_rz%2Fr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 3029241 

[...truncated too long message...]

r, HdfsWriteToMultipleCollectionsTest, TestManagedSchemaFieldResource, DistributedFacetPivotLongTailTest, DeleteLastCustomShardedReplicaTest, SuggestComponentContextFilterQueryTest, CursorMarkTest, TestQuerySenderNoQuery, HttpPartitionTest, TestCursorMarkWithoutUniqueKey, TestRestoreCore, TermVectorComponentTest, TestConfigReload, ChaosMonkeyNothingIsSafeTest, SharedFSAutoReplicaFailoverUtilsTest, TestDistribDocBasedVersion, TestSolrCoreProperties, MinimalSchemaTest, SolrIndexConfigTest, TestExceedMaxTermLength, OpenExchangeRatesOrgProviderTest, TestOrdValues, SortByFunctionTest, TestSchemaVersionResource, MoreLikeThisHandlerTest, SuggesterFSTTest, TestManagedSchema, DocumentBuilderTest, CollectionsAPIAsyncDistributedZkTest, TestManagedSynonymFilterFactory, BlockJoinFacetDistribTest, IndexBasedSpellCheckerTest, CollectionsAPIDistributedZkTest, NumericFieldsTest, SolrCmdDistributorTest, TestQueryUtils, ZkCLITest, TestQuerySenderListener, ChaosMonkeySafeLeaderTest, UnloadDistributedZkTest, OpenCloseCoreStressTest, TestDistributedGrouping, TestFaceting, TestHashPartitioner, DistributedSpellCheckComponentTest, ZkControllerTest, TestRealTimeGet, TestStressReorder, TestJoin, TestReload, TestStressVersions, HardAutoCommitTest, DistributedTermsComponentTest, SimpleFacetsTest, TestSolr4Spatial, SolrCoreTest, QueryElevationComponentTest, PeerSyncTest, BadIndexSchemaTest, ConvertedLegacyTest, TestSort, TestFiltering, TestLazyCores, TestBadConfig, HighlighterTest, DistributedQueryElevationComponentTest, CurrencyFieldXmlFileTest, SolrIndexSplitterTest, SimplePostToolTest, TestCoreDiscovery, AnalysisAfterCoreReloadTest, TestExtendedDismaxParser, SolrRequestParserTest, DocValuesTest, SuggesterTSTTest, SuggesterTest, TestTrie, SchemaVersionSpecificBehaviorTest, TestAtomicUpdateErrorCases, TestWordDelimiterFilterFactory, StatelessScriptUpdateProcessorFactoryTest, DistanceFunctionTest, TestSurroundQueryParser, StandardRequestHandlerTest, TestReversedWildcardFilterFactory, DocumentAnalysisRequestHandlerTest, TestOmitPositions, RequestHandlersTest, FieldAnalysisRequestHandlerTest, FastVectorHighlighterTest, TestJmxIntegration, MBeansHandlerTest, TestAnalyzedSuggestions, JsonLoaderTest, CSVRequestHandlerTest, SearchHandlerTest, HighlighterConfigTest, SOLR749Test, UpdateParamsTest, AlternateDirectoryTest, TestSolrIndexConfig, TestMergePolicyConfig, TestDocSet, TestFuzzyAnalyzedSuggestions, ExternalFileFieldSortTest, NotRequiredUniqueKeyTest, TestLuceneMatchVersion, TestCodecSupport, TestDFRSimilarityFactory, TestLMDirichletSimilarityFactory, TestBM25SimilarityFactory, PluginInfoTest, DateMathParserTest, TestSystemIdResolver, PrimUtilsTest, DateFieldTest, TestRTGBase, SolrTestCaseJ4Test, TestCrossCoreJoin, TestDistributedMissingSort, TestJettySolrRunner, ActionThrottleTest, AssignTest, AsyncMigrateRouteKeyTest, CloudExitableDirectoryReaderTest, ConcurrentDeleteAndCreateCollectionTest, DistribDocExpirationUpdateProcessorTest, OverseerStatusTest, RecoveryAfterSoftCommitTest, ReplicationFactorTest, SSLMigrationTest, ShardSplitTest, HdfsChaosMonkeySafeLeaderTest]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=HdfsChaosMonkeySafeLeaderTest -Dtests.seed=EA3B02E14E63D573 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=et -Dtests.timezone=Navajo -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s J1 | HdfsChaosMonkeySafeLeaderTest (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest: 
   [junit4]    >    1) Thread[id=15765, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:136)
   [junit4]    >    2) Thread[id=15764, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:136)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([EA3B02E14E63D573]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=15764, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
   [junit4]    >         at java.lang.Integer.getChars(Integer.java:372)
   [junit4]    >         at java.lang.Integer.toString(Integer.java:332)
   [junit4]    >         at java.lang.Integer.toString(Integer.java:739)
   [junit4]    >         at org.apache.solr.client.solrj.util.ClientUtils.writeVal(ClientUtils.java:164)
   [junit4]    >         at org.apache.solr.client.solrj.util.ClientUtils.writeXML(ClientUtils.java:132)
   [junit4]    >         at org.apache.solr.client.solrj.request.UpdateRequest.writeXML(UpdateRequest.java:413)
   [junit4]    >         at org.apache.solr.client.solrj.request.UpdateRequest.getXML(UpdateRequest.java:329)
   [junit4]    >         at org.apache.solr.client.solrj.request.RequestWriter.getContentStream(RequestWriter.java:67)
   [junit4]    >         at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:95)
   [junit4]    >         at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:105)
   [junit4]    >         at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:356)
   [junit4]    >         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:239)
   [junit4]    >         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    >         at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
   [junit4]    >         at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:167)
   [junit4]    >         at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:177)
   [junit4]    >         at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:116)
   [junit4]    >    2) Thread[id=15765, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
   [junit4]    >         at java.lang.Integer.getChars(Integer.java:372)
   [junit4]    >         at java.lang.Integer.toString(Integer.java:332)
   [junit4]    >         at java.lang.Integer.toString(Integer.java:739)
   [junit4]    >         at org.apache.solr.client.solrj.util.ClientUtils.writeVal(ClientUtils.java:164)
   [junit4]    >         at org.apache.solr.client.solrj.util.ClientUtils.writeXML(ClientUtils.java:132)
   [junit4]    >         at org.apache.solr.client.solrj.request.UpdateRequest.writeXML(UpdateRequest.java:413)
   [junit4]    >         at org.apache.solr.client.solrj.request.UpdateRequest.getXML(UpdateRequest.java:329)
   [junit4]    >         at org.apache.solr.client.solrj.request.RequestWriter.getContentStream(RequestWriter.java:67)
   [junit4]    >         at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:95)
   [junit4]    >         at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:105)
   [junit4]    >         at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:356)
   [junit4]    >         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:239)
   [junit4]    >         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    >         at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
   [junit4]    >         at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:167)
   [junit4]    >         at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:177)
   [junit4]    >         at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:116)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([EA3B02E14E63D573]:0)
   [junit4] Completed [449/562 (1!)] on J1 in 102.53s, 1 test, 1 failure, 2 errors <<< FAILURES!

[...truncated 355 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1444: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1000: There were test failures: 562 suites (7 ignored), 2231 tests, 2 suite-level errors, 1 failure, 435 ignored (11 assumptions) [seed: EA3B02E14E63D573]

Total time: 185 minutes 56 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1060 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1060/

14 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([5794DB3F94C5B69D]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([5794DB3F94C5B69D]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=16958, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=16958, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.OutOfMemoryError: Java heap space
	at __randomizedtesting.SeedInfo.seed([5794DB3F94C5B69D]:0)
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:881)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=17877, name=zkCallback-924-thread-13, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=17877, name=zkCallback-924-thread-13, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.OutOfMemoryError: Java heap space


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=16961, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=16961, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:36290/lhl/sa, http://127.0.0.1:56062/lhl/sa, http://127.0.0.1:50409/lhl/sa, http://127.0.0.1:42459/lhl/sa, http://127.0.0.1:33539/lhl/sa]
	at __randomizedtesting.SeedInfo.seed([5794DB3F94C5B69D]:0)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:891)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:36290/lhl/sa, http://127.0.0.1:56062/lhl/sa, http://127.0.0.1:50409/lhl/sa, http://127.0.0.1:42459/lhl/sa, http://127.0.0.1:33539/lhl/sa]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:881)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:50409/lhl/sa
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	... 7 more
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
	... 11 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=16956, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=16956, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:36290/lhl/sa, http://127.0.0.1:56062/lhl/sa, http://127.0.0.1:50409/lhl/sa, http://127.0.0.1:42459/lhl/sa, http://127.0.0.1:33539/lhl/sa]
	at __randomizedtesting.SeedInfo.seed([5794DB3F94C5B69D]:0)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:891)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:36290/lhl/sa, http://127.0.0.1:56062/lhl/sa, http://127.0.0.1:50409/lhl/sa, http://127.0.0.1:42459/lhl/sa, http://127.0.0.1:33539/lhl/sa]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:881)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:50409/lhl/sa
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	... 7 more
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
	... 11 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=16960, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=16960, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:36290/lhl/sa, http://127.0.0.1:56062/lhl/sa, http://127.0.0.1:50409/lhl/sa, http://127.0.0.1:42459/lhl/sa, http://127.0.0.1:33539/lhl/sa]
	at __randomizedtesting.SeedInfo.seed([5794DB3F94C5B69D]:0)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:891)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:36290/lhl/sa, http://127.0.0.1:56062/lhl/sa, http://127.0.0.1:50409/lhl/sa, http://127.0.0.1:42459/lhl/sa, http://127.0.0.1:33539/lhl/sa]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:881)
Caused by: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://127.0.0.1:50409/lhl/sa
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:589)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	... 7 more
Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to 127.0.0.1:50409 timed out
	at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:119)
	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
	at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
	... 11 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=16959, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=16959, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:36290/lhl/sa, http://127.0.0.1:56062/lhl/sa, http://127.0.0.1:50409/lhl/sa, http://127.0.0.1:42459/lhl/sa, http://127.0.0.1:33539/lhl/sa]
	at __randomizedtesting.SeedInfo.seed([5794DB3F94C5B69D]:0)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:891)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:36290/lhl/sa, http://127.0.0.1:56062/lhl/sa, http://127.0.0.1:50409/lhl/sa, http://127.0.0.1:42459/lhl/sa, http://127.0.0.1:33539/lhl/sa]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:881)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:50409/lhl/sa
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	... 7 more
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
	... 11 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=16957, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=16957, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:36290/lhl/sa, http://127.0.0.1:56062/lhl/sa, http://127.0.0.1:50409/lhl/sa, http://127.0.0.1:42459/lhl/sa, http://127.0.0.1:33539/lhl/sa]
	at __randomizedtesting.SeedInfo.seed([5794DB3F94C5B69D]:0)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:891)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:36290/lhl/sa, http://127.0.0.1:56062/lhl/sa, http://127.0.0.1:50409/lhl/sa, http://127.0.0.1:42459/lhl/sa, http://127.0.0.1:33539/lhl/sa]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:50409/lhl/sa
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	... 5 more
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
	... 9 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=16573, name=OverseerThreadFactory-2864-thread-2, state=RUNNABLE, group=Overseer collection creation process.]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=16573, name=OverseerThreadFactory-2864-thread-2, state=RUNNABLE, group=Overseer collection creation process.]
Caused by: java.lang.OutOfMemoryError: Java heap space


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=16876, name=SyncThread:0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=16876, name=SyncThread:0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.SecurityException: java.lang.System#exit(11) calls are not allowed because they terminate the test runner's JVM.
	at __randomizedtesting.SeedInfo.seed([5794DB3F94C5B69D]:0)
	at org.apache.lucene.util.TestSecurityManager$1.run(TestSecurityManager.java:86)
	at org.apache.lucene.util.TestSecurityManager$1.run(TestSecurityManager.java:53)
	at java.security.AccessController.doPrivileged(Native Method)
	at org.apache.lucene.util.TestSecurityManager.checkExit(TestSecurityManager.java:53)
	at java.lang.Runtime.exit(Runtime.java:107)
	at java.lang.System.exit(System.java:962)
	at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:185)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=16398, name=OverseerCollectionConfigSetProcessor-95115405963362308-127.0.0.1:36290_lhl%2Fsa-n_0000000000, state=RUNNABLE, group=Overseer collection creation process.]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=16398, name=OverseerCollectionConfigSetProcessor-95115405963362308-127.0.0.1:36290_lhl%2Fsa-n_0000000000, state=RUNNABLE, group=Overseer collection creation process.]
Caused by: java.lang.OutOfMemoryError: Java heap space


FAILED:  org.apache.solr.cloud.HttpPartitionTest.test

Error Message:
Unexpected achieved replication factor expected:<2> but was:<1>

Stack Trace:
java.lang.AssertionError: Unexpected achieved replication factor expected:<2> but was:<1>
	at __randomizedtesting.SeedInfo.seed([5794DB3F94C5B69D:DFC0E4E53A39DB65]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.HttpPartitionTest.testMinRf(HttpPartitionTest.java:212)
	at org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:104)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:965)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:940)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest

Error Message:
ObjectTracker found 1 object(s) that were not released!!! [HdfsTransactionLog]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 1 object(s) that were not released!!! [HdfsTransactionLog]
	at __randomizedtesting.SeedInfo.seed([5794DB3F94C5B69D]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:228)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10328 lines...]
   [junit4] Suite: org.apache.solr.cloud.HttpPartitionTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_5794DB3F94C5B69D-001/init-core-data-001
   [junit4]   2> 402607 INFO  (SUITE-HttpPartitionTest-seed#[5794DB3F94C5B69D]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 402614 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 402614 INFO  (Thread-5456) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 402614 INFO  (Thread-5456) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 402714 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.ZkTestServer start zk server on port:32769
   [junit4]   2> 402714 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 402715 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 402718 INFO  (zkCallback-154-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@b2986c5 name:ZooKeeperConnection Watcher:127.0.0.1:32769 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 402719 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 402719 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 402719 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 402722 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 402722 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 402724 INFO  (zkCallback-155-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@52c2cc70 name:ZooKeeperConnection Watcher:127.0.0.1:32769/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 402724 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 402724 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 402724 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 402726 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 402730 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 402731 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 402733 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 402733 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 402736 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 402736 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 402738 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 402738 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 402739 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 402740 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 402741 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 402742 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 402743 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 402744 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 402746 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 402746 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 402748 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 402748 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 402750 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 402751 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 402753 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 402753 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 402755 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 402755 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 403127 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_5794DB3F94C5B69D-001/control-001/cores/collection1
   [junit4]   2> 403130 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 403132 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4fceb7a9{/,null,AVAILABLE}
   [junit4]   2> 403132 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.e.j.s.ServerConnector Started ServerConnector@18fc9969{HTTP/1.1}{127.0.0.1:44670}
   [junit4]   2> 403132 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.e.j.s.Server Started @406329ms
   [junit4]   2> 403133 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, hostPort=53114, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_5794DB3F94C5B69D-001/control-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_5794DB3F94C5B69D-001/tempDir-001/control/data}
   [junit4]   2> 403133 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4f6de641
   [junit4]   2> 403133 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_5794DB3F94C5B69D-001/control-001'
   [junit4]   2> 403133 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 403133 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 403133 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 403134 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 403135 INFO  (zkCallback-156-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@365668b name:ZooKeeperConnection Watcher:127.0.0.1:32769/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 403135 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 403136 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 403137 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 403137 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_5794DB3F94C5B69D-001/control-001/solr.xml
   [junit4]   2> 403165 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_5794DB3F94C5B69D-001/control-001/cores
   [junit4]   2> 403165 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.CoreContainer New CoreContainer 754941518
   [junit4]   2> 403166 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_5794DB3F94C5B69D-001/control-001]
   [junit4]   2> 403166 WARN  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_5794DB3F94C5B69D-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_5794DB3F94C5B69D-001/control-001/lib
   [junit4]   2> 403166 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 403167 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 403168 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 403168 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 403168 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:32769/solr
   [junit4]   2> 403168 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 403168 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 403169 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 403170 INFO  (zkCallback-158-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@54aaa5e name:ZooKeeperConnection Watcher:127.0.0.1:32769 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 403171 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 403171 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 403173 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [n:127.0.0.1:53114_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 403174 INFO  (zkCallback-159-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@291c2833 name:ZooKeeperConnection Watcher:127.0.0.1:32769/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 403174 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [n:127.0.0.1:53114_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 403176 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [n:127.0.0.1:53114_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 403178 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [n:127.0.0.1:53114_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 403181 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [n:127.0.0.1:53114_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 403184 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [n:127.0.0.1:53114_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 403188 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [n:127.0.0.1:53114_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 403190 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [n:127.0.0.1:53114_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 403192 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [n:127.0.0.1:53114_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 403194 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [n:127.0.0.1:53114_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 403195 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [n:127.0.0.1:53114_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 403196 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [n:127.0.0.1:53114_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53114_
   [junit4]   2> 403197 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [n:127.0.0.1:53114_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:53114_
   [junit4]   2> 403199 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [n:127.0.0.1:53114_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 403201 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [n:127.0.0.1:53114_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 403202 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [n:127.0.0.1:53114_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 403203 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [n:127.0.0.1:53114_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95115331493625860-127.0.0.1:53114_-n_0000000000
   [junit4]   2> 403203 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [n:127.0.0.1:53114_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:53114_
   [junit4]   2> 403203 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [n:127.0.0.1:53114_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 403205 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [n:127.0.0.1:53114_    ] o.a.s.c.Overseer Overseer (id=95115331493625860-127.0.0.1:53114_-n_0000000000) starting
   [junit4]   2> 403207 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [n:127.0.0.1:53114_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 403215 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [n:127.0.0.1:53114_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 403217 INFO  (OverseerCollectionConfigSetProcessor-95115331493625860-127.0.0.1:53114_-n_0000000000) [n:127.0.0.1:53114_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 403217 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [n:127.0.0.1:53114_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 403218 INFO  (OverseerStateUpdate-95115331493625860-127.0.0.1:53114_-n_0000000000) [n:127.0.0.1:53114_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 403249 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [n:127.0.0.1:53114_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 403249 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [n:127.0.0.1:53114_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 403250 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [n:127.0.0.1:53114_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_5794DB3F94C5B69D-001/control-001/cores
   [junit4]   2> 403251 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [n:127.0.0.1:53114_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {dataDir=data/, name=collection1, loadOnStartup=true, configSetProperties=configsetprops.json, collection=control_collection, coreNodeName=, schema=schema.xml, transient=false, config=solrconfig.xml, shard=}
   [junit4]   2> 403251 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [n:127.0.0.1:53114_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_5794DB3F94C5B69D-001/control-001/cores/collection1
   [junit4]   2> 403251 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [n:127.0.0.1:53114_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 403253 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 403253 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 403255 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [n:127.0.0.1:53114_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 403255 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [n:127.0.0.1:53114_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 403256 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 403256 INFO  (OverseerStateUpdate-95115331493625860-127.0.0.1:53114_-n_0000000000) [n:127.0.0.1:53114_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:53114_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:53114"} current state version: 0
   [junit4]   2> 403257 INFO  (OverseerStateUpdate-95115331493625860-127.0.0.1:53114_-n_0000000000) [n:127.0.0.1:53114_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:53114_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:53114"}
   [junit4]   2> 403258 INFO  (OverseerStateUpdate-95115331493625860-127.0.0.1:53114_-n_0000000000) [n:127.0.0.1:53114_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 403258 INFO  (OverseerStateUpdate-95115331493625860-127.0.0.1:53114_-n_0000000000) [n:127.0.0.1:53114_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 403260 INFO  (zkCallback-159-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 404257 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 404257 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 404258 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 404258 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 404259 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 404259 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_5794DB3F94C5B69D-001/control-001/cores/collection1'
   [junit4]   2> 404259 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 404259 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 404261 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 404273 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 404329 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 404362 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 404380 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 404632 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 404640 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 404642 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 404654 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 404657 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 404660 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 404661 WARN  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 404661 WARN  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 404661 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 404661 WARN  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 404662 WARN  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 404663 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 404663 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 404663 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 404663 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_5794DB3F94C5B69D-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_5794DB3F94C5B69D-001/control-001/cores/collection1/data/]
   [junit4]   2> 404663 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@171ac096
   [junit4]   2> 404664 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_5794DB3F94C5B69D-001/control-001/cores/collection1/data
   [junit4]   2> 404665 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_5794DB3F94C5B69D-001/control-001/cores/collection1/data/index/
   [junit4]   2> 404665 WARN  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_5794DB3F94C5B69D-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 404665 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_5794DB3F94C5B69D-001/control-001/cores/collection1/data/index
   [junit4]   2> 404666 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=28, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6681872322933218]
   [junit4]   2> 404668 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(NIOFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_5794DB3F94C5B69D-001/index-NIOFSDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@67c54f30; maxCacheMB=0.1201171875 maxMergeSizeMB=0.638671875)),segFN=segments_1,generation=1}
   [junit4]   2> 404668 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 404668 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_5794DB3F94C5B69D-001/control-001/cores/collection1/data/
   [junit4]   2> 404669 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_5794DB3F94C5B69D-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 404674 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 404674 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 404674 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 404674 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 404674 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 404674 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 404675 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 404675 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 404675 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 404675 WARN  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 404678 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 404679 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 404681 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 404682 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 404686 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/segments,/admin/plugins,/schema,/update/json/docs,/update,/admin/properties,/admin/system,/dump,/admin/file,/config,/update/json,/update/csv,/admin/ping,/replication,/admin/mbeans,/admin/threads,/admin/logging,/get,standard,/admin/luke
   [junit4]   2> 404687 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 404688 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 404688 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 404689 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 404689 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 404689 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@2e55569c
   [junit4]   2> 404698 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(NIOFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_5794DB3F94C5B69D-001/index-NIOFSDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@67c54f30; maxCacheMB=0.1201171875 maxMergeSizeMB=0.638671875)),segFN=segments_1,generation=1}
   [junit4]   2> 404698 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 404699 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@70443256[collection1] main
   [junit4]   2> 404699 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 404700 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 404700 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 404700 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 404700 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 404700 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 404701 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 404701 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 404701 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 404701 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 404702 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 404702 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 404702 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 404702 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1521845306085343232
   [junit4]   2> 404704 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1521845306085343232
   [junit4]   2> 404704 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 404705 INFO  (searcherExecutor-728-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:53114_ x:collection1) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@70443256[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 404707 INFO  (coreLoadExecutor-727-thread-1-processing-n:127.0.0.1:53114_) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 404708 INFO  (coreZkRegister-721-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:53114_ x:collection1) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:53114 collection:control_collection shard:shard1
   [junit4]   2> 404709 INFO  (coreZkRegister-721-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:53114_ x:collection1) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 404716 INFO  (coreZkRegister-721-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:53114_ x:collection1) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95115331493625860-core_node1-n_0000000000
   [junit4]   2> 404716 INFO  (coreZkRegister-721-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:53114_ x:collection1) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 404718 INFO  (OverseerStateUpdate-95115331493625860-127.0.0.1:53114_-n_0000000000) [n:127.0.0.1:53114_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 404720 INFO  (coreZkRegister-721-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:53114_ x:collection1) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 404720 INFO  (coreZkRegister-721-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:53114_ x:collection1) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 404720 INFO  (coreZkRegister-721-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:53114_ x:collection1) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:53114/collection1/
   [junit4]   2> 404720 INFO  (coreZkRegister-721-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:53114_ x:collection1) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 404720 INFO  (coreZkRegister-721-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:53114_ x:collection1) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:53114/collection1/ has no replicas
   [junit4]   2> 404721 INFO  (coreZkRegister-721-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:53114_ x:collection1) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 404723 INFO  (coreZkRegister-721-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:53114_ x:collection1) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 404725 INFO  (OverseerStateUpdate-95115331493625860-127.0.0.1:53114_-n_0000000000) [n:127.0.0.1:53114_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:53114",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 404727 INFO  (coreZkRegister-721-thread-1-processing-s:shard1 r:core_node1 c:control_collection n:127.0.0.1:53114_ x:collection1) [n:127.0.0.1:53114_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:53114/collection1/ shard1
   [junit4]   2> 404765 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 404767 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 404768 INFO  (zkCallback-161-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@59f4795f name:ZooKeeperConnection Watcher:127.0.0.1:32769/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 404768 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 404768 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 404768 INFO  (TEST-HttpPartitionTest.test-seed#[5794DB3F94C5B69D]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2

[...truncated too long message...]

tractMessageParser.parse(AbstractMessageParser.java:261)
   [junit4]    > 	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
   [junit4]    > 	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
   [junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
   [junit4]    > 	... 11 moreThrowable #8: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=16957, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:36290/lhl/sa, http://127.0.0.1:56062/lhl/sa, http://127.0.0.1:50409/lhl/sa, http://127.0.0.1:42459/lhl/sa, http://127.0.0.1:33539/lhl/sa]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([5794DB3F94C5B69D]:0)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:891)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:36290/lhl/sa, http://127.0.0.1:56062/lhl/sa, http://127.0.0.1:50409/lhl/sa, http://127.0.0.1:42459/lhl/sa, http://127.0.0.1:33539/lhl/sa]
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:50409/lhl/sa
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	... 5 more
   [junit4]    > Caused by: java.net.SocketTimeoutException: Read timed out
   [junit4]    > 	at java.net.SocketInputStream.socketRead0(Native Method)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:152)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:122)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
   [junit4]    > 	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
   [junit4]    > 	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
   [junit4]    > 	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
   [junit4]    > 	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
   [junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
   [junit4]    > 	... 9 moreThrowable #9: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=16573, name=OverseerThreadFactory-2864-thread-2, state=RUNNABLE, group=Overseer collection creation process.]
   [junit4]    > Caused by: java.lang.OutOfMemoryError: Java heap spaceThrowable #10: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=16876, name=SyncThread:0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: java.lang.SecurityException: java.lang.System#exit(11) calls are not allowed because they terminate the test runner's JVM.
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([5794DB3F94C5B69D]:0)
   [junit4]    > 	at org.apache.lucene.util.TestSecurityManager$1.run(TestSecurityManager.java:86)
   [junit4]    > 	at org.apache.lucene.util.TestSecurityManager$1.run(TestSecurityManager.java:53)
   [junit4]    > 	at java.security.AccessController.doPrivileged(Native Method)
   [junit4]    > 	at org.apache.lucene.util.TestSecurityManager.checkExit(TestSecurityManager.java:53)
   [junit4]    > 	at java.lang.Runtime.exit(Runtime.java:107)
   [junit4]    > 	at java.lang.System.exit(System.java:962)
   [junit4]    > 	at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:185)Throwable #11: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=16398, name=OverseerCollectionConfigSetProcessor-95115405963362308-127.0.0.1:36290_lhl%2Fsa-n_0000000000, state=RUNNABLE, group=Overseer collection creation process.]
   [junit4]    > Caused by: java.lang.OutOfMemoryError: Java heap space
   [junit4] Completed [561/562 (3!)] on J1 in 12133.90s, 1 test, 12 errors <<< FAILURES!

[...truncated 4 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151228_231253_940.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid7573.hprof ...
   [junit4] Heap dump file created [681177107 bytes in 20.210 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 13 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1456: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1010: There were test failures: 562 suites (7 ignored), 2229 tests, 12 suite-level errors, 1 error, 1 failure, 37 ignored (18 assumptions) [seed: 5794DB3F94C5B69D]

Total time: 388 minutes 52 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1059 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1059/

1 tests failed.
FAILED:  org.apache.lucene.search.TestGeoPointQuery.testRandomBig

Error Message:
CheckIndex failed

Stack Trace:
java.lang.RuntimeException: CheckIndex failed
	at __randomizedtesting.SeedInfo.seed([D0E5AA4BAFF5B4BB:57B2D7C43EACC83B]:0)
	at org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:283)
	at org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:265)
	at org.apache.lucene.store.BaseDirectoryWrapper.close(BaseDirectoryWrapper.java:46)
	at org.apache.lucene.util.IOUtils.close(IOUtils.java:97)
	at org.apache.lucene.util.IOUtils.close(IOUtils.java:84)
	at org.apache.lucene.util.BaseGeoPointTestCase.verify(BaseGeoPointTestCase.java:766)
	at org.apache.lucene.util.BaseGeoPointTestCase.doTestRandom(BaseGeoPointTestCase.java:399)
	at org.apache.lucene.util.BaseGeoPointTestCase.testRandomBig(BaseGeoPointTestCase.java:327)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 8583 lines...]
   [junit4] Suite: org.apache.lucene.search.TestGeoPointQuery
   [junit4]   1> CheckIndex failed
   [junit4]   1> Segments file=segments_1 numSegments=1 version=5.5.0 id=33ism8q1tcour8gqg0m2s0ic0 format=
   [junit4]   1>   1 of 1: name=_m maxDoc=1025769
   [junit4]   1>     version=5.5.0
   [junit4]   1>     id=33ism8q1tcour8gqg0m2s0ibz
   [junit4]   1>     codec=Asserting(Lucene54)
   [junit4]   1>     compound=true
   [junit4]   1>     numFiles=3
   [junit4]   1>     size (MB)=57.194
   [junit4]   1>     diagnostics = {os.arch=amd64, os.version=3.13.0-52-generic, java.runtime.version=1.7.0_80-b15, java.vm.version=24.80-b11, mergeMaxNumSegments=1, os=Linux, java.version=1.7.0_80, mergeFactor=2, java.vendor=Oracle Corporation, timestamp=1451255563397, source=merge, lucene.version=5.5.0}
   [junit4]   1>     no deletions
   [junit4]   1>     test: open reader.........FAILED
   [junit4]   1>     WARNING: exorciseIndex() would remove reference to this segment; full exception:
   [junit4]   1> java.lang.OutOfMemoryError: Java heap space
   [junit4]   1> 	at org.apache.lucene.util.ArrayUtil.grow(ArrayUtil.java:354)
   [junit4]   1> 	at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectField.<init>(DirectPostingsFormat.java:363)
   [junit4]   1> 	at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectFields.<init>(DirectPostingsFormat.java:131)
   [junit4]   1> 	at org.apache.lucene.codecs.memory.DirectPostingsFormat.fieldsProducer(DirectPostingsFormat.java:115)
   [junit4]   1> 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:261)
   [junit4]   1> 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
   [junit4]   1> 	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:104)
   [junit4]   1> 	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
   [junit4]   1> 	at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:649)
   [junit4]   1> 	at org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:278)
   [junit4]   1> 	at org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:265)
   [junit4]   1> 	at org.apache.lucene.store.BaseDirectoryWrapper.close(BaseDirectoryWrapper.java:46)
   [junit4]   1> 	at org.apache.lucene.util.IOUtils.close(IOUtils.java:97)
   [junit4]   1> 	at org.apache.lucene.util.IOUtils.close(IOUtils.java:84)
   [junit4]   1> 	at org.apache.lucene.util.BaseGeoPointTestCase.verify(BaseGeoPointTestCase.java:766)
   [junit4]   1> 	at org.apache.lucene.util.BaseGeoPointTestCase.doTestRandom(BaseGeoPointTestCase.java:399)
   [junit4]   1> 	at org.apache.lucene.util.BaseGeoPointTestCase.testRandomBig(BaseGeoPointTestCase.java:327)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   1> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   1> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   1> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   1> 
   [junit4]   1> WARNING: 1 broken segments (containing 1025769 documents) detected
   [junit4]   1> Took 11.553 sec total.
   [junit4]   1> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestGeoPointQuery -Dtests.method=testRandomBig -Dtests.seed=D0E5AA4BAFF5B4BB -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=mk_MK -Dtests.timezone=Europe/Zagreb -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR    419s J1 | TestGeoPointQuery.testRandomBig <<<
   [junit4]    > Throwable #1: java.lang.RuntimeException: CheckIndex failed
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([D0E5AA4BAFF5B4BB:57B2D7C43EACC83B]:0)
   [junit4]    > 	at org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:283)
   [junit4]    > 	at org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:265)
   [junit4]    > 	at org.apache.lucene.store.BaseDirectoryWrapper.close(BaseDirectoryWrapper.java:46)
   [junit4]    > 	at org.apache.lucene.util.IOUtils.close(IOUtils.java:97)
   [junit4]    > 	at org.apache.lucene.util.IOUtils.close(IOUtils.java:84)
   [junit4]    > 	at org.apache.lucene.util.BaseGeoPointTestCase.verify(BaseGeoPointTestCase.java:766)
   [junit4]    > 	at org.apache.lucene.util.BaseGeoPointTestCase.doTestRandom(BaseGeoPointTestCase.java:399)
   [junit4]    > 	at org.apache.lucene.util.BaseGeoPointTestCase.testRandomBig(BaseGeoPointTestCase.java:327)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/sandbox/test/J1/temp/lucene.search.TestGeoPointQuery_D0E5AA4BAFF5B4BB-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {point=PostingsFormat(name=Direct), id=PostingsFormat(name=LuceneFixedGap), string=FSTOrd50}, docValues:{point=DocValuesFormat(name=Memory), id=DocValuesFormat(name=Asserting)}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=mk_MK, timezone=Europe/Zagreb
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=63900648,total=477102080
   [junit4]   2> NOTE: All tests run in this JVM: [TestJakartaRegexpCapabilities, DuplicateFilterTest, TestGeoPointQuery]
   [junit4] Completed [20/20 (1!)] on J1 in 1230.24s, 23 tests, 1 error <<< FAILURES!

[...truncated 1 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/sandbox/test/temp/junit4-J1-20151227_221815_330.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid12503.hprof ...
   [junit4] Heap dump file created [549490151 bytes in 4.476 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 10 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:472: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2252: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1456: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1010: There were test failures: 20 suites, 142 tests, 1 error [seed: D0E5AA4BAFF5B4BB]

Total time: 152 minutes 36 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1058 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1058/

1 tests failed.
FAILED:  org.apache.solr.core.TestArbitraryIndexDir.testLoadNewIndexDir

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([6F1F3CA24A4AB83A:8645879AD4D32892]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:749)
	at org.apache.solr.core.TestArbitraryIndexDir.testLoadNewIndexDir(TestArbitraryIndexDir.java:107)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=*[count(//doc)=1]
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
</response>

	request was:rows=20&start=0&qt=standard&q=id:2&version=2.2
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:742)
	... 41 more




Build Log:
[...truncated 11261 lines...]
   [junit4] Suite: org.apache.solr.core.TestArbitraryIndexDir
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.TestArbitraryIndexDir_6F1F3CA24A4AB83A-001/init-core-data-001
   [junit4]   2> 2877922 INFO  (SUITE-TestArbitraryIndexDir-seed#[6F1F3CA24A4AB83A]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false)
   [junit4]   2> 2877926 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testLoadNewIndexDir
   [junit4]   2> 2877927 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 2877927 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1'
   [junit4]   2> 2877927 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2877927 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.SolrResourceLoader using system property solr.solr.home: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr
   [junit4]   2> 2877927 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 2877927 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 2877928 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 2877959 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2877979 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 2878021 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2878028 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.s.IndexSchema [null] Schema name=test
   [junit4]   2> 2878405 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2878410 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2878411 WARN  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 2878417 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2878420 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2878422 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2878422 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.SolrResourceLoader using system property solr.solr.home: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr
   [junit4]   2> 2878423 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr'
   [junit4]   2> 2878423 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2878423 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.SolrResourceLoader using system property solr.solr.home: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr
   [junit4]   2> 2878423 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.CoreContainer New CoreContainer 1012182096
   [junit4]   2> 2878423 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr]
   [junit4]   2> 2878423 WARN  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/lib
   [junit4]   2> 2878423 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 600000,connTimeout : 60000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2878427 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=30000&connTimeout=30000&retry=true
   [junit4]   2> 2878428 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2878428 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2878428 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2878428 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2878429 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {transient=false, collection=collection1, shard=shard1, dataDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.TestArbitraryIndexDir_6F1F3CA24A4AB83A-001/init-core-data-001, name=collection1, config=solrconfig.xml, loadOnStartup=true, configSetProperties=configsetprops.json, schema=schema12.xml}
   [junit4]   2> 2878430 INFO  (coreLoadExecutor-6131-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1'
   [junit4]   2> 2878430 INFO  (coreLoadExecutor-6131-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2878430 INFO  (coreLoadExecutor-6131-thread-1) [    ] o.a.s.c.SolrResourceLoader using system property solr.solr.home: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr
   [junit4]   2> 2878430 INFO  (coreLoadExecutor-6131-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 2878431 INFO  (coreLoadExecutor-6131-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 2878431 INFO  (coreLoadExecutor-6131-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 2878466 INFO  (coreLoadExecutor-6131-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2878487 INFO  (coreLoadExecutor-6131-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 2878530 INFO  (coreLoadExecutor-6131-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2878682 INFO  (coreLoadExecutor-6131-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2879036 INFO  (coreLoadExecutor-6131-thread-1) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2879037 INFO  (coreLoadExecutor-6131-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2879038 WARN  (coreLoadExecutor-6131-thread-1) [    ] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 2879044 INFO  (coreLoadExecutor-6131-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2879058 INFO  (coreLoadExecutor-6131-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2879063 INFO  (coreLoadExecutor-6131-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1'
   [junit4]   2> 2879063 INFO  (coreLoadExecutor-6131-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1
   [junit4]   2> 2879063 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockFSDirectoryFactory
   [junit4]   2> 2879063 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.TestArbitraryIndexDir_6F1F3CA24A4AB83A-001/init-core-data-001/]
   [junit4]   2> 2879064 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4aee0373
   [junit4]   2> 2879064 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.TestArbitraryIndexDir_6F1F3CA24A4AB83A-001/init-core-data-001
   [junit4]   2> 2879064 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.TestArbitraryIndexDir_6F1F3CA24A4AB83A-001/init-core-data-001/index/
   [junit4]   2> 2879065 WARN  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.TestArbitraryIndexDir_6F1F3CA24A4AB83A-001/init-core-data-001/index' doesn't exist. Creating new index...
   [junit4]   2> 2879065 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.TestArbitraryIndexDir_6F1F3CA24A4AB83A-001/init-core-data-001/index
   [junit4]   2> 2879065 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=5, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 2879081 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(NIOFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.TestArbitraryIndexDir_6F1F3CA24A4AB83A-001/init-core-data-001/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5ba26e03; maxCacheMB=0.0625 maxMergeSizeMB=0.443359375)),segFN=segments_1,generation=1}
   [junit4]   2> 2879081 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2879095 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2879095 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2879095 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 2879095 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 2879095 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2879096 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2879096 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 2879096 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 2879096 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2879096 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2879096 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2879096 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2879099 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2879101 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2879102 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2879107 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.TestArbitraryIndexDir_6F1F3CA24A4AB83A-001/init-core-data-001/
   [junit4]   2> 2879108 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2879122 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 2879125 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /search-facet-invariants,/update,/schema,spellCheckCompRH1,/admin/ping,spellCheckCompRH,standard,/admin/logging,/update/csv,/debug/dump,spellCheckWithWordbreak_Direct,/terms,/admin/file,/get,spellCheckWithWordbreak,lazy,/search-facet-def,/admin/mbeans,/admin/threads,defaults,spellCheckCompRH_Direct,/replication,tvrh,/config,mock,/admin/segments,/admin/system,/admin/plugins,/update/json/docs,/admin/luke,mltrh,dismax,/update/json,/admin/properties,/mlt
   [junit4]   2> 2879127 WARN  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.c.SolrCore solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
   [junit4]   2> 2879129 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2879143 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2879143 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2879144 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=48, maxMergeAtOnceExplicit=26, maxMergedSegmentMB=51.0048828125, floorSegmentMB=1.087890625, forceMergeDeletesPctAllowed=16.2419654704005, segmentsPerTier=37.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 2879151 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(NIOFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.TestArbitraryIndexDir_6F1F3CA24A4AB83A-001/init-core-data-001/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5ba26e03; maxCacheMB=0.0625 maxMergeSizeMB=0.443359375)),segFN=segments_1,generation=1}
   [junit4]   2> 2879151 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2879151 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@69284d4a[collection1] main
   [junit4]   2> 2879152 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
   [junit4]   2> 2879152 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 2879152 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2879152 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 2879158 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 2879279 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2879279 INFO  (searcherExecutor-6132-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 2879279 INFO  (searcherExecutor-6132-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 2879279 INFO  (searcherExecutor-6132-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 2879279 INFO  (searcherExecutor-6132-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 2879280 INFO  (searcherExecutor-6132-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 2879280 INFO  (coreLoadExecutor-6131-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2879280 INFO  (searcherExecutor-6132-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 2879280 INFO  (searcherExecutor-6132-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 2879281 INFO  (searcherExecutor-6132-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 2879281 INFO  (searcherExecutor-6132-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 2879282 INFO  (searcherExecutor-6132-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@69284d4a[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2879283 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 2879285 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 1
   [junit4]   2> 2879323 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2879360 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(NIOFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.TestArbitraryIndexDir_6F1F3CA24A4AB83A-001/init-core-data-001/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5ba26e03; maxCacheMB=0.0625 maxMergeSizeMB=0.443359375)),segFN=segments_1,generation=1}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(NIOFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.TestArbitraryIndexDir_6F1F3CA24A4AB83A-001/init-core-data-001/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5ba26e03; maxCacheMB=0.0625 maxMergeSizeMB=0.443359375)),segFN=segments_2,generation=2}
   [junit4]   2> 2879360 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.SolrDeletionPolicy newest commit generation = 2
   [junit4]   2> 2879362 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.SolrCore New index directory detected: old=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.TestArbitraryIndexDir_6F1F3CA24A4AB83A-001/init-core-data-001/index/ new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.TestArbitraryIndexDir_6F1F3CA24A4AB83A-001/init-core-data-001/index_temp
   [junit4]   2> 2879364 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.s.SolrIndexSearcher Opening Searcher@6ffe6a18[collection1] main
   [junit4]   2> 2879365 INFO  (searcherExecutor-6132-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6ffe6a18[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(5.5.0):C1)))}
   [junit4]   2> 2879365 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 2879366 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 42
   [junit4]   2> 2879367 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1'
   [junit4]   2> 2879367 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2879367 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.SolrResourceLoader using system property solr.solr.home: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr
   [junit4]   2> 2879367 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 2879367 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 2879367 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 2879404 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2879424 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 2879467 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2879475 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2879706 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2879708 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2879709 WARN  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 2879714 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2879717 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2879720 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1'
   [junit4]   2> 2879721 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    ] o.a.s.c.CoreContainer Reloading SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1
   [junit4]   2> 2879721 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.TestArbitraryIndexDir_6F1F3CA24A4AB83A-001/init-core-data-001/]
   [junit4]   2> 2879721 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4aee0373
   [junit4]   2> 2879722 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.TestArbitraryIndexDir_6F1F3CA24A4AB83A-001/init-core-data-001/index_temp
   [junit4]   2> 2879728 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2879728 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2879728 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 2879728 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 2879728 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.TestArbitraryIndexDir_6F1F3CA24A4AB83A-001/init-core-data-001/
   [junit4]   2> 2879728 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2879729 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2879729 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 2879729 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 2879729 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2879729 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.DirectoryFactory Found 1 old index directories to clean-up under /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.TestArbitraryIndexDir_6F1F3CA24A4AB83A-001/init-core-data-001/
   [junit4]   2> 2879729 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Cannot delete directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.TestArbitraryIndexDir_6F1F3CA24A4AB83A-001/init-core-data-001/index as it is still being referenced in the cache!
   [junit4]   2> 2879729 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2879729 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.DirectoryFactory Delete old index directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.TestArbitraryIndexDir_6F1F3CA24A4AB83A-001/init-core-data-001/index failed.
   [junit4]   2> 2879729 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2879730 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2879732 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2879734 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2879735 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2879736 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2879737 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.TestArbitraryIndexDir_6F1F3CA24A4AB83A-001/init-core-data-001/index_temp
   [junit4]   2> 2879742 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 2879745 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /search-facet-invariants,/update,/schema,spellCheckCompRH1,/admin/ping,spellCheckCompRH,standard,/admin/logging,/update/csv,/debug/dump,spellCheckWithWordbreak_Direct,/terms,/admin/file,/get,spellCheckWithWordbreak,lazy,/search-facet-def,/admin/mbeans,/admin/threads,defaults,spellCheckCompRH_Direct,/replication,tvrh,/config,mock,/admin/segments,/admin/system,/admin/plugins,/update/json/docs,/admin/luke,mltrh,dismax,/update/json,/admin/properties,/mlt
   [junit4]   2> 2879746 WARN  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.c.SolrCore solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
   [junit4]   2> 2879747 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2879747 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2879747 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2879748 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@46eba7db[collection1] main
   [junit4]   2> 2879749 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
   [junit4]   2> 2879749 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 2879749 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2879749 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 2879750 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 2879753 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2879753 INFO  (searcherExecutor-6136-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 2879754 INFO  (searcherExecutor-6136-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 2879754 INFO  (searcherExecutor-6136-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 2879754 INFO  (searcherExecutor-6136-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 2879754 INFO  (searcherExecutor-6136-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 2879754 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.u.DefaultSolrCoreState Closing old IndexWriter... core=collection1
   [junit4]   2> 2879755 INFO  (searcherExecutor-6136-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 2879755 INFO  (searcherExecutor-6136-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 2879755 INFO  (searcherExecutor-6136-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 2879756 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@70681768
   [junit4]   2> 2879756 INFO  (searcherExecutor-6136-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 2879757 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NIOFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.TestArbitraryIndexDir_6F1F3CA24A4AB83A-001/init-core-data-001/index_temp lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5c70f3ea),segFN=segments_1,generation=1}
   [junit4]   2> 2879757 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2879757 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 2879767 INFO  (searcherExecutor-6136-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@46eba7db[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(5.5.0):C1)))}
   [junit4]   2> 2879774 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@29144065[collection1] main
   [junit4]   2> 2879774 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.c.CoreContainer replacing core: collection1
   [junit4]   2> 2879774 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@a806811
   [junit4]   2> 2879774 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.u.DirectUpdateHandler2 closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=1,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 2879775 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 2879782 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={rows=20&start=0&qt=standard&q=id:2&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 2879783 ERROR (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.SolrTestCaseJ4 REQUEST FAILED: xpath=*[count(//doc)=1]
   [junit4]   2> 	xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]   2> <response>
   [junit4]   2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
   [junit4]   2> </response>
   [junit4]   2> 
   [junit4]   2> 	request was:rows=20&start=0&qt=standard&q=id:2&version=2.2
   [junit4]   2> 2879784 ERROR (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.SolrTestCaseJ4 REQUEST FAILED: rows=20&start=0&qt=standard&q=id:2&version=2.2:java.lang.RuntimeException: REQUEST FAILED: xpath=*[count(//doc)=1]
   [junit4]   2> 	xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]   2> <response>
   [junit4]   2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
   [junit4]   2> </response>
   [junit4]   2> 
   [junit4]   2> 	request was:rows=20&start=0&qt=standard&q=id:2&version=2.2
   [junit4]   2> 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:742)
   [junit4]   2> 	at org.apache.solr.core.TestArbitraryIndexDir.testLoadNewIndexDir(TestArbitraryIndexDir.java:107)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 2879784 INFO  (TEST-TestArbitraryIndexDir.testLoadNewIndexDir-seed#[6F1F3CA24A4AB83A]) [    x:collection1] o.a.s.SolrTestCaseJ4 ###Ending testLoadNewIndexDir
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestArbitraryIndexDir -Dtests.method=testLoadNewIndexDir -Dtests.seed=6F1F3CA24A4AB83A -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=ro_RO -Dtests.timezone=Europe/Kiev -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   1.86s J2 | TestArbitraryIndexDir.testLoadNewIndexDir <<<
   [junit4]    > Throwable #1: java.lang.RuntimeException: Exception during query
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([6F1F3CA24A4AB83A:8645879AD4D32892]:0)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:749)
   [junit4]    > 	at org.apache.solr.core.TestArbitraryIndexDir.testLoadNewIndexDir(TestArbitraryIndexDir.java:107)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]    > Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=*[count(//doc)=1]
   [junit4]    > 	xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]    > <response>
   [junit4]    > <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
   [junit4]    > </response>
   [junit4]    > 	request was:rows=20&start=0&qt=standard&q=id:2&version=2.2
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:742)
   [junit4]    > 	... 41 more
   [junit4]   2> 2879787 INFO  (SUITE-TestArbitraryIndexDir-seed#[6F1F3CA24A4AB83A]-worker) [    x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> 2879787 INFO  (SUITE-TestArbitraryIndexDir-seed#[6F1F3CA24A4AB83A]-worker) [    x:collection1] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1012182096
   [junit4]   2> 2879787 INFO  (SUITE-TestArbitraryIndexDir-seed#[6F1F3CA24A4AB83A]-worker) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@501bf3c8
   [junit4]   2> 2879788 INFO  (SUITE-TestArbitraryIndexDir-seed#[6F1F3CA24A4AB83A]-worker) [    x:collection1] o.a.s.u.DirectUpdateHandler2 closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 2879788 INFO  (SUITE-TestArbitraryIndexDir-seed#[6F1F3CA24A4AB83A]-worker) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 2879788 INFO  (SUITE-TestArbitraryIndexDir-seed#[6F1F3CA24A4AB83A]-worker) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 2879788 INFO  (SUITE-TestArbitraryIndexDir-seed#[6F1F3CA24A4AB83A]-worker) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 2879789 INFO  (searcherExecutor-6136-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@29144065[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(5.5.0):c1)))}
   [junit4]   2> 2879789 INFO  (SUITE-TestArbitraryIndexDir-seed#[6F1F3CA24A4AB83A]-worker) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 2879801 INFO  (SUITE-TestArbitraryIndexDir-seed#[6F1F3CA24A4AB83A]-worker) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockFSDirectoryFactory - 3 directories currently being tracked
   [junit4]   2> 2879802 INFO  (SUITE-TestArbitraryIndexDir-seed#[6F1F3CA24A4AB83A]-worker) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.TestArbitraryIndexDir_6F1F3CA24A4AB83A-001/init-core-data-001/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.TestArbitraryIndexDir_6F1F3CA24A4AB83A-001/init-core-data-001/index;done=false>>]
   [junit4]   2> 2879802 INFO  (SUITE-TestArbitraryIndexDir-seed#[6F1F3CA24A4AB83A]-worker) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.TestArbitraryIndexDir_6F1F3CA24A4AB83A-001/init-core-data-001/index
   [junit4]   2> 2879820 INFO  (SUITE-TestArbitraryIndexDir-seed#[6F1F3CA24A4AB83A]-worker) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.TestArbitraryIndexDir_6F1F3CA24A4AB83A-001/init-core-data-001/index_temp [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.TestArbitraryIndexDir_6F1F3CA24A4AB83A-001/init-core-data-001/index_temp;done=false>>]
   [junit4]   2> 2879820 INFO  (SUITE-TestArbitraryIndexDir-seed#[6F1F3CA24A4AB83A]-worker) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.TestArbitraryIndexDir_6F1F3CA24A4AB83A-001/init-core-data-001/index_temp
   [junit4]   2> 2879826 INFO  (SUITE-TestArbitraryIndexDir-seed#[6F1F3CA24A4AB83A]-worker) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.TestArbitraryIndexDir_6F1F3CA24A4AB83A-001/init-core-data-001 [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.TestArbitraryIndexDir_6F1F3CA24A4AB83A-001/init-core-data-001;done=false>>]
   [junit4]   2> 2879826 INFO  (SUITE-TestArbitraryIndexDir-seed#[6F1F3CA24A4AB83A]-worker) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.TestArbitraryIndexDir_6F1F3CA24A4AB83A-001/init-core-data-001
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.core.TestArbitraryIndexDir_6F1F3CA24A4AB83A-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54), sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=ro_RO, timezone=Europe/Kiev
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=43175752,total=500170752
   [junit4]   2> NOTE: All tests run in this JVM: [DistribDocExpirationUpdateProcessorTest, SimpleMLTQParserTest, RecoveryZkTest, SuggesterTSTTest, SuggesterTest, TestFuzzyAnalyzedSuggestions, TestPerFieldSimilarity, TestPHPSerializedResponseWriter, BadIndexSchemaTest, TestReload, ClusterStateTest, ExitableDirectoryReaderTest, TestCharFilters, TestUtils, DirectUpdateHandlerTest, TestDistribIDF, TestSha256AuthenticationProvider, RuleEngineTest, TestMissingGroups, AtomicUpdatesTest, AnalyticsMergeStrategyTest, TestRecoveryHdfs, CoreAdminHandlerTest, TestIndexSearcher, PluginInfoTest, LukeRequestHandlerTest, TestClassNameShortening, CleanupOldIndexTest, TestConfigSetsAPIZkFailure, SliceStateTest, TestZkChroot, TestRandomMergePolicy, SimpleCollectionCreateDeleteTest, DistributedFacetPivotLargeTest, HdfsCollectionsAPIDistributedZkTest, XmlUpdateRequestHandlerTest, TestTolerantSearch, ActionThrottleTest, TestStandardQParsers, TestRandomCollapseQParserPlugin, TestRebalanceLeaders, DeleteReplicaTest, DateMathParserTest, TestElisionMultitermQuery, EnumFieldTest, UnloadDistributedZkTest, QueryParsingTest, RequestHandlersTest, TestManagedResourceStorage, TestSortByMinMaxFunction, TestComplexPhraseQParserPlugin, TestClusterStateMutator, HdfsUnloadDistributedZkTest, MigrateRouteKeyTest, TestPostingsSolrHighlighter, TestGroupingSearch, TestObjectReleaseTracker, TestSolrQueryResponse, StatsComponentTest, HardAutoCommitTest, TestReloadAndDeleteDocs, BasicDistributedZk2Test, BlockDirectoryTest, ScriptEngineTest, RemoteQueryErrorTest, TestSolrQueryParser, TestRemoveLastDynamicCopyField, ConcurrentDeleteAndCreateCollectionTest, TestAuthorizationFramework, NoCacheHeaderTest, DistributedFacetPivotSmallTest, TestWriterPerf, TermVectorComponentDistributedTest, TestExtendedDismaxParser, TestManagedStopFilterFactory, TestDistributedStatsComponentCardinality, TestSearchPerf, TestConfigSets, TestAddFieldRealTimeGet, RegexBoostProcessorTest, TestCloudManagedSchema, CircularListTest, OverriddenZkACLAndCredentialsProvidersTest, TestInfoStreamLogging, TestStressRecovery, TestReqParamsAPI, BigEndianAscendingWordSerializerTest, TestRecovery, SimpleFacetsTest, OverseerTest, TestReplicationHandler, AutoCommitTest, TestDistributedGrouping, TestFaceting, TestRealTimeGet, TestStressReorder, TestJoin, TestStressVersions, DistributedTermsComponentTest, TestCoreContainer, SolrCoreTest, QueryElevationComponentTest, TestSort, TestFunctionQuery, TestLazyCores, ShowFileRequestHandlerTest, CurrencyFieldOpenExchangeTest, CurrencyFieldXmlFileTest, AnalysisAfterCoreReloadTest, SpellCheckCollatorTest, TestFoldingMultitermQuery, TestStressLucene, PolyFieldTest, SolrCoreCheckLockOnStartupTest, DirectUpdateHandlerOptimizeTest, DefaultValueUpdateProcessorTest, TestRemoteStreaming, DistanceFunctionTest, XsltUpdateRequestHandlerTest, DisMaxRequestHandlerTest, DirectSolrSpellCheckerTest, DocumentAnalysisRequestHandlerTest, TermsComponentTest, TestIndexingPerformance, MoreLikeThisHandlerTest, TestArbitraryIndexDir]
   [junit4] Completed [349/558 (1!)] on J2 in 1.94s, 1 test, 1 error <<< FAILURES!

[...truncated 548 lines...]
   [junit4] JVM J0: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J0-20151226_235120_978.sysout
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid10336.hprof ...
   [junit4] Heap dump file created [615492942 bytes in 6.650 secs]
   [junit4] <<< JVM J0: EOF ----

   [junit4] JVM J0: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J0-20151226_235120_978.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] 
   [junit4] Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "TEST-CollectionsAPIDistributedZkTest.test-seed#[6F1F3CA24A4AB83A]-SendThread(127.0.0.1:50878)"
   [junit4] WARN: Unhandled exception in event serialization. -> java.lang.OutOfMemoryError: Java heap space
   [junit4] <<< JVM J0: EOF ----

[...truncated 156 lines...]
   [junit4] ERROR: JVM J0 ended with an exception, command line: /x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/jre/bin/java -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps -XX:MaxPermSize=192m -ea -esa -Dtests.prefix=tests -Dtests.seed=6F1F3CA24A4AB83A -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.luceneMatchVersion=5.5.0 -Dtests.cleanthreads=perClass -Djava.util.logging.config.file=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/tools/junit4/logging.properties -Dtests.nightly=true -Dtests.weekly=false -Dtests.monster=false -Dtests.slow=true -Dtests.asserts=true -Dtests.multiplier=2 -DtempDir=./temp -Djava.io.tmpdir=./temp -Djunit4.tempDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp -Dcommon.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene -Dclover.db.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/clover/db -Djava.security.policy=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/tools/junit4/solr-tests.policy -Dtests.LUCENE_VERSION=5.5.0 -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Djdk.map.althashing.threshold=0 -Djunit4.childvm.cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0 -Djunit4.childvm.id=0 -Djunit4.childvm.count=3 -Dtests.leaveTemporary=false -Dtests.filterstacks=true -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -Dfile.encoding=US-ASCII -classpath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-test-framework/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/test-framework/lib/junit4-ant-2.3.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/test-framework/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/codecs/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-solrj/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/common/lucene-analyzers-common-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/codecs/lucene-codecs-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/backward-codecs/lucene-backward-codecs-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/highlighter/lucene-highlighter-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/memory/lucene-memory-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/misc/lucene-misc-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/spatial/lucene-spatial-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/expressions/lucene-expressions-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/suggest/lucene-suggest-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/grouping/lucene-grouping-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/queries/lucene-queries-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/queryparser/lucene-queryparser-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/join/lucene-join-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/sandbox/lucene-sandbox-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/antlr4-runtime-4.5.1-1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/asm-5.0.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/asm-commons-5.0.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-cli-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-codec-1.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-configuration-1.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-exec-1.3.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-fileupload-1.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-lang-2.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/concurrentlinkedhashmap-lru-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/dom4j-1.6.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/guava-14.0.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/hadoop-annotations-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/hadoop-auth-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/hadoop-common-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/hadoop-hdfs-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/hppc-0.7.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/htrace-core-3.0.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/jackson-core-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/jackson-dataformat-smile-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/joda-time-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/log4j-1.2.17.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/org.restlet-2.3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/org.restlet.ext.servlet-2.3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/protobuf-java-2.5.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/slf4j-log4j12-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/spatial4j-0.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/t-digest-3.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/commons-io-2.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/httpclient-4.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/httpcore-4.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/httpmime-4.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/noggit-0.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/slf4j-api-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/stax2-api-3.1.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/woodstox-core-asl-4.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/zookeeper-3.4.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/javax.servlet-api-3.1.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-continuation-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-deploy-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-http-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-io-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-jmx-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-rewrite-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-security-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-server-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-servlet-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-servlets-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-util-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-webapp-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-xml-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/example/example-DIH/solr/db/lib/derby-10.9.1.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/test-framework/lib/junit-4.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/test-framework/lib/randomizedtesting-runner-2.3.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/antlr-2.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/antlr4-runtime-4.5.1-1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-core-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-core-api-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-core-avl-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-core-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-i18n-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptor-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-admin-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.2.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J0-20151226_235120_978.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J0-20151226_235120_978.suites -stdin
   [junit4] ERROR: JVM J0 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1456: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1010: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 249 minutes 23 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1057 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1057/

10 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([E657957F422809DC]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=20263, name=SessionTracker, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=20263, name=SessionTracker, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=20180, name=OverseerThreadFactory-2949-thread-3, state=RUNNABLE, group=Overseer collection creation process.]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=20180, name=OverseerThreadFactory-2949-thread-3, state=RUNNABLE, group=Overseer collection creation process.]
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=20346, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=20346, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=20347, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=20347, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:37344, http://127.0.0.1:37043, http://127.0.0.1:34227, http://127.0.0.1:53220, http://127.0.0.1:33894]
	at __randomizedtesting.SeedInfo.seed([E657957F422809DC]:0)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:891)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:37344, http://127.0.0.1:37043, http://127.0.0.1:34227, http://127.0.0.1:53220, http://127.0.0.1:33894]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37344: KeeperErrorCode = Session expired for /overseer/collection-queue-work/qnr-0000000054
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	... 5 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=20350, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=20350, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:37344, http://127.0.0.1:37043, http://127.0.0.1:34227, http://127.0.0.1:53220, http://127.0.0.1:33894]
	at __randomizedtesting.SeedInfo.seed([E657957F422809DC]:0)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:891)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:37344, http://127.0.0.1:37043, http://127.0.0.1:34227, http://127.0.0.1:53220, http://127.0.0.1:33894]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:881)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37344: KeeperErrorCode = Session expired for /overseer/collection-queue-work/qnr-0000000048
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	... 7 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=20349, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=20349, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:37344, http://127.0.0.1:37043, http://127.0.0.1:34227, http://127.0.0.1:53220, http://127.0.0.1:33894]
	at __randomizedtesting.SeedInfo.seed([E657957F422809DC]:0)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:891)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:37344, http://127.0.0.1:37043, http://127.0.0.1:34227, http://127.0.0.1:53220, http://127.0.0.1:33894]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:881)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37344: KeeperErrorCode = Session expired for /overseer/collection-queue-work/qnr-0000000049
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	... 7 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=20348, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=20348, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:37344, http://127.0.0.1:37043, http://127.0.0.1:34227, http://127.0.0.1:53220, http://127.0.0.1:33894]
	at __randomizedtesting.SeedInfo.seed([E657957F422809DC]:0)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:891)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:37344, http://127.0.0.1:37043, http://127.0.0.1:34227, http://127.0.0.1:53220, http://127.0.0.1:33894]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:881)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37344: KeeperErrorCode = Session expired for /overseer/collection-queue-work/qnr-0000000050
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	... 7 more


FAILED:  org.apache.solr.core.TestDynamicLoading.testDynamicLoading

Error Message:
Could not get expected value  'X val changed' for path 'x' full output: {   "responseHeader":{     "status":0,     "QTime":0},   "params":{"wt":"json"},   "context":{     "path":"/test1",     "webapp":"",     "httpMethod":"GET"},   "class":"org.apache.solr.core.BlobStoreTestRequestHandler",   "x":"X val"}

Stack Trace:
java.lang.AssertionError: Could not get expected value  'X val changed' for path 'x' full output: {
  "responseHeader":{
    "status":0,
    "QTime":0},
  "params":{"wt":"json"},
  "context":{
    "path":"/test1",
    "webapp":"",
    "httpMethod":"GET"},
  "class":"org.apache.solr.core.BlobStoreTestRequestHandler",
  "x":"X val"}
	at __randomizedtesting.SeedInfo.seed([E657957F422809DC:3E1AB828B5F5AC7C]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.core.TestSolrConfigHandler.testForResponseElement(TestSolrConfigHandler.java:458)
	at org.apache.solr.core.TestDynamicLoading.testDynamicLoading(TestDynamicLoading.java:257)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:965)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:940)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([E657957F422809DC]:0)




Build Log:
[...truncated 10302 lines...]
   [junit4] Suite: org.apache.solr.core.TestDynamicLoading
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_E657957F422809DC-001/init-core-data-001
   [junit4]   2> 348709 INFO  (SUITE-TestDynamicLoading-seed#[E657957F422809DC]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 348710 INFO  (SUITE-TestDynamicLoading-seed#[E657957F422809DC]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 348727 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 348732 INFO  (Thread-990) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 348733 INFO  (Thread-990) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 348832 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.ZkTestServer start zk server on port:38434
   [junit4]   2> 348833 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 348833 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 348836 INFO  (zkCallback-268-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@f0c8e90 name:ZooKeeperConnection Watcher:127.0.0.1:38434 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 348836 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 348836 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 348836 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 348840 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 348840 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 348841 INFO  (zkCallback-269-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@32788314 name:ZooKeeperConnection Watcher:127.0.0.1:38434/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 348841 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 348842 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 348842 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 348844 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 348845 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 348846 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 348848 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 348849 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 348851 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 348852 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 348854 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 348854 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 348856 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 348856 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 348857 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 348858 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 348859 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 348859 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 348861 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 348861 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 348863 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 348863 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 348864 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 348865 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 348866 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 348867 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 348868 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 348868 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 349182 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_E657957F422809DC-001/control-001/cores/collection1
   [junit4]   2> 349183 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 349185 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@70f9aae{/,null,AVAILABLE}
   [junit4]   2> 349189 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.e.j.s.ServerConnector Started ServerConnector@5d1dce7a{HTTP/1.1}{127.0.0.1:33610}
   [junit4]   2> 349189 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.e.j.s.Server Started @351757ms
   [junit4]   2> 349189 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_E657957F422809DC-001/tempDir-001/control/data, hostPort=33610, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_E657957F422809DC-001/control-001/cores}
   [junit4]   2> 349190 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 349190 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_E657957F422809DC-001/control-001'
   [junit4]   2> 349190 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 349190 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 349190 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 349191 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 349192 INFO  (zkCallback-270-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4f540fad name:ZooKeeperConnection Watcher:127.0.0.1:38434/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 349193 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 349193 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 349194 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 349194 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_E657957F422809DC-001/control-001/solr.xml
   [junit4]   2> 349211 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_E657957F422809DC-001/control-001/cores
   [junit4]   2> 349212 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.CoreContainer New CoreContainer 1704335193
   [junit4]   2> 349212 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_E657957F422809DC-001/control-001]
   [junit4]   2> 349212 WARN  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_E657957F422809DC-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_E657957F422809DC-001/control-001/lib
   [junit4]   2> 349213 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 349214 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 349215 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 349215 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 349215 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38434/solr
   [junit4]   2> 349215 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 349215 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 349216 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 349217 INFO  (zkCallback-272-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@584e2da name:ZooKeeperConnection Watcher:127.0.0.1:38434 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 349218 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 349218 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 349220 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:33610_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 349221 INFO  (zkCallback-273-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@29899b25 name:ZooKeeperConnection Watcher:127.0.0.1:38434/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 349221 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:33610_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 349222 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:33610_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 349224 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:33610_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 349227 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:33610_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 349229 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:33610_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 349231 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:33610_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 349233 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:33610_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 349234 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:33610_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 349236 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:33610_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 349237 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:33610_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 349238 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:33610_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33610_
   [junit4]   2> 349238 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:33610_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:33610_
   [junit4]   2> 349240 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:33610_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 349241 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:33610_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 349242 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:33610_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 349243 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:33610_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95099030373859332-127.0.0.1:33610_-n_0000000000
   [junit4]   2> 349243 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:33610_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33610_
   [junit4]   2> 349243 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:33610_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 349245 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:33610_    ] o.a.s.c.Overseer Overseer (id=95099030373859332-127.0.0.1:33610_-n_0000000000) starting
   [junit4]   2> 349246 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:33610_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 349252 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:33610_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 349252 INFO  (OverseerCollectionConfigSetProcessor-95099030373859332-127.0.0.1:33610_-n_0000000000) [n:127.0.0.1:33610_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 349252 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:33610_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 349253 INFO  (OverseerStateUpdate-95099030373859332-127.0.0.1:33610_-n_0000000000) [n:127.0.0.1:33610_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 349270 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:33610_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 349270 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:33610_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 349271 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:33610_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_E657957F422809DC-001/control-001/cores
   [junit4]   2> 349273 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:33610_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, loadOnStartup=true, config=solrconfig.xml, collection=control_collection, shard=, schema=schema.xml, transient=false, configSetProperties=configsetprops.json, dataDir=data/, coreNodeName=}
   [junit4]   2> 349273 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:33610_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_E657957F422809DC-001/control-001/cores/collection1
   [junit4]   2> 349273 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:33610_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 349274 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:33610_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 349274 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:33610_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 349274 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 349274 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 349276 INFO  (OverseerStateUpdate-95099030373859332-127.0.0.1:33610_-n_0000000000) [n:127.0.0.1:33610_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33610",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:33610_"} current state version: 0
   [junit4]   2> 349277 INFO  (OverseerStateUpdate-95099030373859332-127.0.0.1:33610_-n_0000000000) [n:127.0.0.1:33610_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33610",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:33610_"}
   [junit4]   2> 349277 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 349277 INFO  (OverseerStateUpdate-95099030373859332-127.0.0.1:33610_-n_0000000000) [n:127.0.0.1:33610_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 349278 INFO  (OverseerStateUpdate-95099030373859332-127.0.0.1:33610_-n_0000000000) [n:127.0.0.1:33610_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 349279 INFO  (zkCallback-273-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 350277 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 350278 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 350278 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 350278 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 350279 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 350279 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_E657957F422809DC-001/control-001/cores/collection1'
   [junit4]   2> 350279 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 350279 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 350281 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 350289 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 350304 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 350337 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 350348 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 350616 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 350626 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 350628 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 350642 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 350647 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 350650 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 350651 WARN  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 350651 WARN  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 350651 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 350651 WARN  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 350652 WARN  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 350653 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 350653 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 350653 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 350653 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_E657957F422809DC-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_E657957F422809DC-001/control-001/cores/collection1/data/]
   [junit4]   2> 350653 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4f7a1892
   [junit4]   2> 350654 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_E657957F422809DC-001/control-001/cores/collection1/data
   [junit4]   2> 350654 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_E657957F422809DC-001/control-001/cores/collection1/data/index/
   [junit4]   2> 350654 WARN  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_E657957F422809DC-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 350655 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_E657957F422809DC-001/control-001/cores/collection1/data/index
   [junit4]   2> 350656 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=30, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 350657 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@74167ad1 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6fd96ae1),segFN=segments_1,generation=1}
   [junit4]   2> 350657 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 350660 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_E657957F422809DC-001/control-001/cores/collection1/data/
   [junit4]   2> 350660 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_E657957F422809DC-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 350668 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 350668 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 350668 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 350668 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 350668 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 350669 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 350669 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 350669 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 350669 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 350669 WARN  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 350672 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 350674 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 350675 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 350676 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 350681 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/file,/update,/replication,/admin/luke,/dump,/admin/segments,/admin/system,/admin/ping,/get,/update/json/docs,/admin/plugins,/admin/logging,standard,/update/csv,/config,/update/json,/admin/mbeans,/admin/threads,/admin/properties,/schema
   [junit4]   2> 350682 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 350683 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 350683 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 350684 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 350684 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 350685 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=11, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=13.0615234375, floorSegmentMB=1.30859375, forceMergeDeletesPctAllowed=19.25491330465318, segmentsPerTier=13.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8719329207811348
   [junit4]   2> 350685 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@74167ad1 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6fd96ae1),segFN=segments_1,generation=1}
   [junit4]   2> 350685 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 350686 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@301658b2[collection1] main
   [junit4]   2> 350686 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 350687 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 350687 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 350687 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 350687 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 350687 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 350687 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 350688 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 350688 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 350688 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 350688 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 350688 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 350688 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 350688 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1521584488026472448
   [junit4]   2> 350689 INFO  (searcherExecutor-675-thread-1-processing-x:collection1 c:control_collection r:core_node1 s:shard1 n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@301658b2[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 350691 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1521584488026472448
   [junit4]   2> 350691 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 350691 INFO  (coreLoadExecutor-674-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 350692 INFO  (coreZkRegister-668-thread-1-processing-x:collection1 c:control_collection r:core_node1 s:shard1 n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:33610 collection:control_collection shard:shard1
   [junit4]   2> 350692 INFO  (coreZkRegister-668-thread-1-processing-x:collection1 c:control_collection r:core_node1 s:shard1 n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 350696 INFO  (coreZkRegister-668-thread-1-processing-x:collection1 c:control_collection r:core_node1 s:shard1 n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95099030373859332-core_node1-n_0000000000
   [junit4]   2> 350696 INFO  (coreZkRegister-668-thread-1-processing-x:collection1 c:control_collection r:core_node1 s:shard1 n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 350698 INFO  (coreZkRegister-668-thread-1-processing-x:collection1 c:control_collection r:core_node1 s:shard1 n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 350698 INFO  (coreZkRegister-668-thread-1-processing-x:collection1 c:control_collection r:core_node1 s:shard1 n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 350698 INFO  (coreZkRegister-668-thread-1-processing-x:collection1 c:control_collection r:core_node1 s:shard1 n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:33610/collection1/
   [junit4]   2> 350698 INFO  (coreZkRegister-668-thread-1-processing-x:collection1 c:control_collection r:core_node1 s:shard1 n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 350698 INFO  (coreZkRegister-668-thread-1-processing-x:collection1 c:control_collection r:core_node1 s:shard1 n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:33610/collection1/ has no replicas
   [junit4]   2> 350699 INFO  (OverseerStateUpdate-95099030373859332-127.0.0.1:33610_-n_0000000000) [n:127.0.0.1:33610_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 350699 INFO  (coreZkRegister-668-thread-1-processing-x:collection1 c:control_collection r:core_node1 s:shard1 n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 350702 INFO  (coreZkRegister-668-thread-1-processing-x:collection1 c:control_collection r:core_node1 s:shard1 n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 350704 INFO  (coreZkRegister-668-thread-1-processing-x:collection1 c:control_collection r:core_node1 s:shard1 n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:33610/collection1/ shard1
   [junit4]   2> 350705 INFO  (OverseerStateUpdate-95099030373859332-127.0.0.1:33610_-n_0000000000) [n:127.0.0.1:33610_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:33610",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 350776 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 350777 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 351166 INFO  (zkCallback-273-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 351167 INFO  (zkCallback-275-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@40fb46db name:ZooKeeperConnection Watcher:127.0.0.1:38434/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 351168 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 351168 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 351168 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 351171 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 351215 INFO  (coreZkRegister-668-thread-1-processing-x:collection1 c:control_collection r:core_node1 s:shard1 n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:33610/collection1/ and leader is http://127.0.0.1:33610/collection1/
   [junit4]   2> 351215 INFO  (coreZkRegister-668-thread-1-processing-x:collection1 c:control_collection r:core_node1 s:shard1 n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:33610
   [junit4]   2> 351215 INFO  (coreZkRegister-668-thread-1-processing-x:collection1 c:control_collection r:core_node1 s:shard1 n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 351215 INFO  (coreZkRegister-668-thread-1-processing-x:collection1 c:control_collection r:core_node1 s:shard1 n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 351215 INFO  (coreZkRegister-668-thread-1-processing-x:collection1 c:control_collection r:core_node1 s:shard1 n:127.0.0.1:33610_) [n:127.0.0.1:33610_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 351217 INFO  (OverseerStateUpdate-95099030373859332-127.0.0.1:33610_-n_0000000000) [n:127.0.0.1:33610_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "base_url":"http://127.0.0.1:33610",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "node_name":"127.0.0.1:33610_"} current state version: 2
   [junit4]   2> 351218 INFO  (OverseerStateUpdate-95099030373859332-127.0.0.1:33610_-n_0000000000) [n:127.0.0.1:33610_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "base_url":"http://127.0.0.1:33610",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "node_name":"127.0.0.1:33610_"}
   [junit4]   2> 351320 INFO  (zkCallback-275-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 351320 INFO  (zkCallback-273-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 351488 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_E657957F422809DC-001/shard-1-001/cores/collection1
   [junit4]   2> 351489 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_E657957F422809DC-001/shard-1-001
   [junit4]   2> 351490 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 351491 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@12876401{/,null,AVAILABLE}
   [junit4]   2> 351492 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.e.j.s.ServerConnector Started ServerConnector@49634e54{HTTP/1.1}{127.0.0.1:51851}
   [junit4]   2> 351492 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.e.j.s.Server Started @354060ms
   [junit4]   2> 351492 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=51851, solrconfig=solrconfig.xml, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_E657957F422809DC-001/tempDir-001/jetty1, hostContext=/, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_E657957F422809DC-001/shard-1-001/cores}
   [junit4]   2> 351492 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 351493 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_E657957F422809DC-001/shard-1-001'
   [junit4]   2> 351493 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 351493 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 351493 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 351494 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 351495 INFO  (zkCallback-276-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1ba68676 name:ZooKeeperConnection Watcher:127.0.0.1:38434/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 351495 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 351495 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 351496 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 351496 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_E657957F422809DC-001/shard-1-001/solr.xml
   [junit4]   2> 351511 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_E657957F422809DC-001/shard-1-001/cores
   [junit4]   2> 351511 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.CoreContainer New CoreContainer 1364394205
   [junit4]   2> 351511 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_E657957F422809DC-001/shard-1-001]
   [junit4]   2> 351511 WARN  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_E657957F422809DC-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_E657957F422809DC-001/shard-1-001/lib
   [junit4]   2> 351512 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 351513 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 351514 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 351514 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 351514 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38434/solr
   [junit4]   2> 351514 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 351514 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 351515 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 351516 INFO  (zkCallback-278-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@16418ccd name:ZooKeeperConnection Watcher:127.0.0.1:38434 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 351517 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 351517 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 351519 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:51851_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 351520 INFO  (zkCallback-279-thread-1-processing-n:127.0.0.1:51851_) [n:127.0.0.1:51851_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5492a5ea name:ZooKeeperConnection Watcher:127.0.0.1:38434/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 351520 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:51851_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 351524 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:51851_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 352529 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:51851_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51851_
   [junit4]   2> 352530 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:51851_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:51851_
   [junit4]   2> 352532 INFO  (zkCallback-279-thread-1-processing-n:127.0.0.1:51851_) [n:127.0.0.1:51851_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 352532 INFO  (zkCallback-275-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 352532 INFO  (zkCallback-273-thread-1-processing-n:127.0.0.1:33610_) [n:127.0.0.1:33610_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 352534 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:51851_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 352534 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:51851_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95099030373859336-127.0.0.1:51851_-n_0000000001
   [junit4]   2> 352535 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:51851_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/95099030373859332-127.0.0.1:33610_-n_0000000000 to know if I could be the leader
   [junit4]   2> 352545 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:51851_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 352545 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:51851_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 352546 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:51851_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_E657957F422809DC-001/shard-1-001/cores
   [junit4]   2> 352548 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:51851_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, collection=collection1, coreNodeName=, shard=, schema=schema.xml, loadOnStartup=true, dataDir=data/, transient=false, config=solrconfig.xml, configSetProperties=configsetprops.json}
   [junit4]   2> 352548 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[E657957F422809DC]) [n:127.0.0.1:51851_    ] o.a.s.c.CoreProperties

[...truncated too long message...]

tExceptionError: Captured an uncaught exception in thread: Thread[id=20350, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:37344, http://127.0.0.1:37043, http://127.0.0.1:34227, http://127.0.0.1:53220, http://127.0.0.1:33894]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([E657957F422809DC]:0)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:891)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:37344, http://127.0.0.1:37043, http://127.0.0.1:34227, http://127.0.0.1:53220, http://127.0.0.1:33894]
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:881)
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37344: KeeperErrorCode = Session expired for /overseer/collection-queue-work/qnr-0000000048
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	... 7 moreThrowable #7: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=20349, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:37344, http://127.0.0.1:37043, http://127.0.0.1:34227, http://127.0.0.1:53220, http://127.0.0.1:33894]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([E657957F422809DC]:0)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:891)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:37344, http://127.0.0.1:37043, http://127.0.0.1:34227, http://127.0.0.1:53220, http://127.0.0.1:33894]
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:881)
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37344: KeeperErrorCode = Session expired for /overseer/collection-queue-work/qnr-0000000049
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	... 7 moreThrowable #8: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=20348, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:37344, http://127.0.0.1:37043, http://127.0.0.1:34227, http://127.0.0.1:53220, http://127.0.0.1:33894]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([E657957F422809DC]:0)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:891)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:37344, http://127.0.0.1:37043, http://127.0.0.1:34227, http://127.0.0.1:53220, http://127.0.0.1:33894]
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:881)
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37344: KeeperErrorCode = Session expired for /overseer/collection-queue-work/qnr-0000000050
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	... 7 more
   [junit4] Completed [558/558 (2!)] on J1 in 7230.02s, 1 test, 9 errors <<< FAILURES!

[...truncated 1 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151226_020813_107.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid3249.hprof ...
   [junit4] Heap dump file created [709909323 bytes in 8.299 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 12 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1456: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1010: There were test failures: 558 suites (7 ignored), 2219 tests, 8 suite-level errors, 1 error, 1 failure, 33 ignored (18 assumptions) [seed: E657957F422809DC]

Total time: 282 minutes 36 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1056 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1056/

1 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=4091, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=4091, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:41008: collection already exists: awholynewstresscollection_collection4_0
	at __randomizedtesting.SeedInfo.seed([AE39A44269F6A5DA]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:881)




Build Log:
[...truncated 10342 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/init-core-data-001
   [junit4]   2> 322845 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[AE39A44269F6A5DA]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 322950 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[AE39A44269F6A5DA]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 322956 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 322957 INFO  (Thread-2423) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 322957 INFO  (Thread-2423) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 323057 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.ZkTestServer start zk server on port:44095
   [junit4]   2> 323057 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 323059 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 323082 INFO  (zkCallback-161-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@18a91dd5 name:ZooKeeperConnection Watcher:127.0.0.1:44095 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 323082 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 323083 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 323083 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 323093 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 323094 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 323102 INFO  (zkCallback-162-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@316b2678 name:ZooKeeperConnection Watcher:127.0.0.1:44095/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 323102 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 323102 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 323102 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 323104 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 323105 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 323106 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 323108 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 323108 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 323110 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 323110 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 323112 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 323112 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 323113 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 323114 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 323115 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 323115 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 323116 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 323117 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 323118 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 323118 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 323119 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 323119 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 323121 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 323121 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 323122 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 323122 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 323124 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 323124 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 323456 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/control-001/cores/collection1
   [junit4]   2> 323458 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 323460 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6d6578c1{/,null,AVAILABLE}
   [junit4]   2> 323463 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.e.j.s.ServerConnector Started ServerConnector@99c2ecf{HTTP/1.1}{127.0.0.1:53991}
   [junit4]   2> 323463 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.e.j.s.Server Started @326607ms
   [junit4]   2> 323463 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, hostPort=53991, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/control-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/tempDir-001/control/data}
   [junit4]   2> 323464 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 323464 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/control-001'
   [junit4]   2> 323464 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 323464 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 323464 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 323467 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 323467 INFO  (zkCallback-163-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@69935f95 name:ZooKeeperConnection Watcher:127.0.0.1:44095/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 323467 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 323468 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 323469 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 323469 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/control-001/solr.xml
   [junit4]   2> 323487 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/control-001/cores
   [junit4]   2> 323487 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.CoreContainer New CoreContainer 1902198075
   [junit4]   2> 323487 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/control-001]
   [junit4]   2> 323487 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/control-001/lib
   [junit4]   2> 323487 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 323489 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 323489 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 323490 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 323490 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44095/solr
   [junit4]   2> 323490 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 323490 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 323491 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 323492 INFO  (zkCallback-165-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@b6d551d name:ZooKeeperConnection Watcher:127.0.0.1:44095 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 323492 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 323492 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 323495 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:53991_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 323495 INFO  (zkCallback-166-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@31e5ebe6 name:ZooKeeperConnection Watcher:127.0.0.1:44095/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 323496 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:53991_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 323497 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:53991_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 323499 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:53991_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 323501 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:53991_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 323503 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:53991_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 323505 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:53991_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 323506 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:53991_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 323508 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:53991_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 323509 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:53991_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 323510 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:53991_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 323511 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:53991_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53991_
   [junit4]   2> 323511 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:53991_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:53991_
   [junit4]   2> 323513 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:53991_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 323514 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:53991_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 323515 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:53991_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 323515 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:53991_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95092791805411332-127.0.0.1:53991_-n_0000000000
   [junit4]   2> 323516 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:53991_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:53991_
   [junit4]   2> 323516 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:53991_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 323517 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:53991_    ] o.a.s.c.Overseer Overseer (id=95092791805411332-127.0.0.1:53991_-n_0000000000) starting
   [junit4]   2> 323519 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:53991_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 323524 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:53991_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 323525 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:53991_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 323525 INFO  (OverseerCollectionConfigSetProcessor-95092791805411332-127.0.0.1:53991_-n_0000000000) [n:127.0.0.1:53991_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 323526 INFO  (OverseerStateUpdate-95092791805411332-127.0.0.1:53991_-n_0000000000) [n:127.0.0.1:53991_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 323567 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:53991_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 323567 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:53991_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 323568 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:53991_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/control-001/cores
   [junit4]   2> 323569 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:53991_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {coreNodeName=, collection=control_collection, config=solrconfig.xml, configSetProperties=configsetprops.json, shard=, dataDir=data/, name=collection1, schema=schema.xml, transient=false, loadOnStartup=true}
   [junit4]   2> 323569 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:53991_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/control-001/cores/collection1
   [junit4]   2> 323569 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:53991_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 323571 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 323571 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 323573 INFO  (OverseerStateUpdate-95092791805411332-127.0.0.1:53991_-n_0000000000) [n:127.0.0.1:53991_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:53991",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:53991_"} current state version: 0
   [junit4]   2> 323574 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 323574 INFO  (OverseerStateUpdate-95092791805411332-127.0.0.1:53991_-n_0000000000) [n:127.0.0.1:53991_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:53991",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:53991_"}
   [junit4]   2> 323574 INFO  (OverseerStateUpdate-95092791805411332-127.0.0.1:53991_-n_0000000000) [n:127.0.0.1:53991_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 323575 INFO  (OverseerStateUpdate-95092791805411332-127.0.0.1:53991_-n_0000000000) [n:127.0.0.1:53991_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 323575 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:53991_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 323575 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:53991_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 323576 INFO  (zkCallback-166-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 324574 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 324574 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 324575 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 324575 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 324578 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 324578 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/control-001/cores/collection1'
   [junit4]   2> 324579 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 324579 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 324580 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 324598 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 324620 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 324704 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 324712 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 325218 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 325239 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 325241 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 325263 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 325266 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 325269 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 325270 WARN  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 325270 WARN  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 325270 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 325271 WARN  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 325271 WARN  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 325278 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 325279 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 325279 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 325279 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/control-001/cores/collection1/data/]
   [junit4]   2> 325279 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ab88e22
   [junit4]   2> 325280 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/control-001/cores/collection1/data
   [junit4]   2> 325280 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/control-001/cores/collection1/data/index/
   [junit4]   2> 325280 WARN  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 325280 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/control-001/cores/collection1/data/index
   [junit4]   2> 325280 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=48, maxMergeAtOnceExplicit=46, maxMergedSegmentMB=1.703125, floorSegmentMB=1.49609375, forceMergeDeletesPctAllowed=7.693163155244856, segmentsPerTier=44.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8797420070410122
   [junit4]   2> 325281 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@69e9c8af lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@73768f7b),segFN=segments_1,generation=1}
   [junit4]   2> 325281 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 325286 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 325287 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 325287 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 325287 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 325287 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 325287 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 325287 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 325288 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 325288 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 325288 WARN  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 325291 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 325292 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 325294 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 325295 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 325297 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/control-001/cores/collection1/data/
   [junit4]   2> 325297 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 325321 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /replication,/admin/file,/admin/threads,/admin/properties,/schema,/admin/segments,/update/json/docs,/admin/luke,/admin/ping,/admin/logging,/get,/admin/plugins,/dump,/config,/admin/mbeans,/update/json,/update,/admin/system,/update/csv,standard
   [junit4]   2> 325322 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 325326 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 325326 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 325327 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 325327 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 325328 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=14, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8447118534851571]
   [junit4]   2> 325328 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@69e9c8af lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@73768f7b),segFN=segments_1,generation=1}
   [junit4]   2> 325328 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 325328 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@38bd336e[collection1] main
   [junit4]   2> 325329 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 325330 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 325330 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 325330 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 325330 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 325330 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 325331 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 325331 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 325331 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 325331 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 325332 INFO  (searcherExecutor-532-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@38bd336e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 325332 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 325332 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 325332 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 325332 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1521484671375900672
   [junit4]   2> 325335 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1521484671375900672
   [junit4]   2> 325335 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 325335 INFO  (coreLoadExecutor-531-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 325336 INFO  (coreZkRegister-525-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:53991 collection:control_collection shard:shard1
   [junit4]   2> 325337 INFO  (coreZkRegister-525-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 325342 INFO  (coreZkRegister-525-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95092791805411332-core_node1-n_0000000000
   [junit4]   2> 325342 INFO  (coreZkRegister-525-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=180000
   [junit4]   2> 325344 INFO  (OverseerStateUpdate-95092791805411332-127.0.0.1:53991_-n_0000000000) [n:127.0.0.1:53991_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 325346 INFO  (zkCallback-166-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 325349 INFO  (coreZkRegister-525-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 325349 INFO  (coreZkRegister-525-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 325349 INFO  (coreZkRegister-525-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:53991/collection1/
   [junit4]   2> 325349 INFO  (coreZkRegister-525-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 325349 INFO  (coreZkRegister-525-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:53991/collection1/ has no replicas
   [junit4]   2> 325350 INFO  (coreZkRegister-525-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 325352 INFO  (coreZkRegister-525-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 325355 INFO  (coreZkRegister-525-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:53991/collection1/ shard1
   [junit4]   2> 325355 INFO  (OverseerStateUpdate-95092791805411332-127.0.0.1:53991_-n_0000000000) [n:127.0.0.1:53991_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:53991",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 325459 INFO  (zkCallback-166-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 325464 INFO  (coreZkRegister-525-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:53991/collection1/ and leader is http://127.0.0.1:53991/collection1/
   [junit4]   2> 325464 INFO  (coreZkRegister-525-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:53991
   [junit4]   2> 325464 INFO  (coreZkRegister-525-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 325464 INFO  (coreZkRegister-525-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 325464 INFO  (coreZkRegister-525-thread-1-processing-c:control_collection r:core_node1 x:collection1 s:shard1 n:127.0.0.1:53991_) [n:127.0.0.1:53991_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 325466 INFO  (OverseerStateUpdate-95092791805411332-127.0.0.1:53991_-n_0000000000) [n:127.0.0.1:53991_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:53991",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "node_name":"127.0.0.1:53991_",
   [junit4]   2>   "core_node_name":"core_node1"} current state version: 3
   [junit4]   2> 325466 INFO  (OverseerStateUpdate-95092791805411332-127.0.0.1:53991_-n_0000000000) [n:127.0.0.1:53991_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:53991",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "node_name":"127.0.0.1:53991_",
   [junit4]   2>   "core_node_name":"core_node1"}
   [junit4]   2> 325569 INFO  (zkCallback-166-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 325577 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 325577 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 325579 INFO  (zkCallback-168-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@77c1c11f name:ZooKeeperConnection Watcher:127.0.0.1:44095/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 325579 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 325579 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 325579 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 325581 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 325582 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 325582 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 325590 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 325610 INFO  (zkCallback-169-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5e02fa0 name:ZooKeeperConnection Watcher:127.0.0.1:44095/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 325610 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 325610 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 325615 INFO  (OverseerStateUpdate-95092791805411332-127.0.0.1:53991_-n_0000000000) [n:127.0.0.1:53991_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 4
   [junit4]   2> 325615 INFO  (OverseerStateUpdate-95092791805411332-127.0.0.1:53991_-n_0000000000) [n:127.0.0.1:53991_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 325718 INFO  (OverseerStateUpdate-95092791805411332-127.0.0.1:53991_-n_0000000000) [n:127.0.0.1:53991_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 326027 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/shard-1-001/cores/collection1
   [junit4]   2> 326028 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/shard-1-001
   [junit4]   2> 326029 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 326084 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6a42514f{/,null,AVAILABLE}
   [junit4]   2> 326084 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.e.j.s.ServerConnector Started ServerConnector@4de9357{HTTP/1.1}{127.0.0.1:45476}
   [junit4]   2> 326084 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.e.j.s.Server Started @329228ms
   [junit4]   2> 326084 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/shard-1-001/cores, hostContext=/, hostPort=45476, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/tempDir-001/jetty1}
   [junit4]   2> 326085 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 326085 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/shard-1-001'
   [junit4]   2> 326085 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 326085 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 326085 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 326086 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 326087 INFO  (zkCallback-170-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@39689892 name:ZooKeeperConnection Watcher:127.0.0.1:44095/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 326087 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 326088 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 326091 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 326092 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/shard-1-001/solr.xml
   [junit4]   2> 326107 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/shard-1-001/cores
   [junit4]   2> 326107 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.CoreContainer New CoreContainer 1353453697
   [junit4]   2> 326107 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/shard-1-001]
   [junit4]   2> 326107 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/shard-1-001/lib
   [junit4]   2> 326107 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 326109 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 326109 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 326109 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 326109 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44095/solr
   [junit4]   2> 326109 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 326109 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 326146 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 326158 INFO  (zkCallback-172-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7c4cb1d3 name:ZooKeeperConnection Watcher:127.0.0.1:44095 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 326159 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 326159 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 326196 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:45476_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 326197 INFO  (zkCallback-173-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@46e75830 name:ZooKeeperConnection Watcher:127.0.0.1:44095/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 326197 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:45476_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 326215 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:45476_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 327226 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:45476_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45476_
   [junit4]   2> 327227 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:45476_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:45476_
   [junit4]   2> 327228 INFO  (zkCallback-168-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 327228 INFO  (zkCallback-166-thread-1-processing-n:127.0.0.1:53991_) [n:127.0.0.1:53991_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 327230 INFO  (zkCallback-173-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 327231 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:45476_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 327231 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:45476_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95092791805411337-127.0.0.1:45476_-n_0000000001
   [junit4]   2> 327232 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:45476_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/95092791805411332-127.0.0.1:53991_-n_0000000000 to know if I could be the leader
   [junit4]   2> 327245 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:45476_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 327245 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:45476_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 327246 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:45476_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/shard-1-001/cores
   [junit4]   2> 327247 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:45476_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {loadOnStartup=true, schema=schema.xml, name=collection1, configSetProperties=configsetprops.json, config=solrconfig.xml, dataDir=data/, collection=collection1, shard=, coreNodeName=, transient=false}
   [junit4]   2> 327247 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:45476_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/shard-1-001/cores/collection1
   [junit4]   2> 327247 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:45476_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 327253 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 327253 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 327255 INFO  (OverseerStateUpdate-95092791805411332-127.0.0.1:53991_-n_0000000000) [n:127.0.0.1:53991_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "base_url":"http://127.0.0.1:45476",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:45476_"} current state version: 4
   [junit4]   2> 327256 INFO  (OverseerStateUpdate-95092791805411332-127.0.0.1:53991_-n_0000000000) [n:127.0.0.1:53991_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "base_url":"http://127.0.0.1:45476",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:45476_"}
   [junit4]   2> 327256 INFO  (OverseerStateUpdate-95092791805411332-127.0.0.1:53991_-n_0000000000) [n:127.0.0.1:53991_    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=2
   [junit4]   2> 327256 INFO  (OverseerStateUpdate-95092791805411332-127.0.0.1:53991_-n_0000000000) [n:127.0.0.1:53991_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 327257 INFO  (OverseerStateUpdate-95092791805411332-127.0.0.1:53991_-n_0000000000) [n:127.0.0.1:53991_    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 327260 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 327260 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch [collection1]
   [junit4]   2> 327261 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:45476_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 327261 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[AE39A44269F6A5DA]) [n:127.0.0.1:45476_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 327261 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for [collection1] ver [1]
   [junit4]   2> 327261 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 327261 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 327261 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 327261 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 327262 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/collection1]
   [junit4]   2> 327262 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/collection1] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 327262 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/shard-1-001/cores/collection1'
   [junit4]   2> 327262 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 327262 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 327265 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 327293 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 327341 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 327372 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 327396 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 327752 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 327767 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 327768 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 327782 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 327786 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 327793 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 327794 WARN  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 327794 WARN  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 327794 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 327795 WARN  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 327795 WARN  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 327796 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 327796 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 327796 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 327797 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/shard-1-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/shard-1-001/cores/collection1/data/]
   [junit4]   2> 327797 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ab88e22
   [junit4]   2> 327798 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/shard-1-001/cores/collection1/data
   [junit4]   2> 327807 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/shard-1-001/cores/collection1/data/index/
   [junit4]   2> 327807 WARN  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/shard-1-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 327808 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_AE39A44269F6A5DA-001/shard-1-001/cores/collection1/data/index
   [junit4]   2> 327808 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=48, maxMergeAtOnceExplicit=46, maxMergedSegmentMB=1.703125, floorSegmentMB=1.49609375, forceMergeDeletesPctAllowed=7.693163155244856, segmentsPerTier=44.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8797420070410122
   [junit4]   2> 327813 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@34ec2877 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7c54b7c5),segFN=segments_1,generation=1}
   [junit4]   2> 327814 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 327831 INFO  (coreLoadExecutor-542-thread-1-processing-n:127.0.0.1:45476_) [n:127.0.0.1:45476_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequ

[...truncated too long message...]

t.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:881)Throwable #3: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=4089, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:41008: collection already exists: awholynewstresscollection_collection2_0
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([AE39A44269F6A5DA]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:881)Throwable #4: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=4092, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:41008: collection already exists: awholynewstresscollection_collection5_0
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([AE39A44269F6A5DA]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:881)Throwable #5: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=4088, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:41008: collection already exists: awholynewstresscollection_collection1_0
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([AE39A44269F6A5DA]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:881)Throwable #6: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=4087, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:41008: Could not find collection : awholynewstresscollection_collection0_0
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([AE39A44269F6A5DA]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)
   [junit4]   2> 702042 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[AE39A44269F6A5DA]-worker) [n:127.0.0.1:43376_ c:addReplicaColl s:shard1 r:core_node7 x:propertyDotName] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=DefaultSimilarity, locale=th_TH, timezone=Asia/Ho_Chi_Minh
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=174332536,total=511705088
   [junit4]   2> NOTE: All tests run in this JVM: [HdfsBasicDistributedZk2Test, TestCollapseQParserPlugin, RecoveryAfterSoftCommitTest, TestUniqueKeyFieldResource, ChaosMonkeyNothingIsSafeTest, CursorMarkTest, TestExactStatsCache, SoftAutoCommitTest, ConvertedLegacyTest, TestTrieFacet, FacetPivotSmallTest, TestPartialUpdateDeduplication, SolrIndexConfigTest, CollectionsAPIDistributedZkTest]
   [junit4] Completed [80/558 (1!)] on J1 in 379.97s, 1 test, 1 error <<< FAILURES!

[...truncated 1567 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1456: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1010: There were test failures: 558 suites (7 ignored), 2219 tests, 1 error, 33 ignored (18 assumptions) [seed: AE39A44269F6A5DA]

Total time: 196 minutes 37 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1055 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1055/

2 tests failed.
FAILED:  org.apache.solr.cloud.CloudExitableDirectoryReaderTest.test

Error Message:
No live SolrServers available to handle this request:[http://127.0.0.1:47131/phpp/collection1]

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:47131/phpp/collection1]
	at __randomizedtesting.SeedInfo.seed([DDD71E7F89D6D30C:558321A5272ABEF4]:0)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.queryServer(AbstractFullDistribZkTestBase.java:1378)
	at org.apache.solr.cloud.CloudExitableDirectoryReaderTest.assertPartialResults(CloudExitableDirectoryReaderTest.java:101)
	at org.apache.solr.cloud.CloudExitableDirectoryReaderTest.doTimeoutTests(CloudExitableDirectoryReaderTest.java:73)
	at org.apache.solr.cloud.CloudExitableDirectoryReaderTest.test(CloudExitableDirectoryReaderTest.java:52)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:965)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:940)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:47131/phpp/collection1: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	... 51 more


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=8444, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=8444, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:34039/_u: Could not find collection : awholynewstresscollection_collection4_3
	at __randomizedtesting.SeedInfo.seed([DDD71E7F89D6D30C]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)




Build Log:
[...truncated 10508 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DDD71E7F89D6D30C-001/init-core-data-001
   [junit4]   2> 209849 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[DDD71E7F89D6D30C]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false)
   [junit4]   2> 209850 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[DDD71E7F89D6D30C]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_u/
   [junit4]   2> 209856 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 209856 INFO  (Thread-7308) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 209856 INFO  (Thread-7308) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 209956 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.ZkTestServer start zk server on port:53816
   [junit4]   2> 209957 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 209958 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 209962 INFO  (zkCallback-115-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@35933cf5 name:ZooKeeperConnection Watcher:127.0.0.1:53816 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 209963 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 209963 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 209963 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 209968 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 209969 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 209972 INFO  (zkCallback-116-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@76947386 name:ZooKeeperConnection Watcher:127.0.0.1:53816/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 209972 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 209973 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 209973 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 209975 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 209977 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 209979 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 209980 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 209981 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 209984 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 209984 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 209986 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 209987 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 209989 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 209989 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 209991 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 209991 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 209993 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 209993 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 209996 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 209996 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 209999 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 209999 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 210003 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 210003 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 210005 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 210005 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 210008 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 210008 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 210012 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 210012 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 210014 INFO  (zkCallback-117-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@958a682 name:ZooKeeperConnection Watcher:127.0.0.1:53816/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 210014 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 210014 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 210017 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 210017 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 210019 INFO  (zkCallback-118-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@9347bea name:ZooKeeperConnection Watcher:127.0.0.1:53816 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 210019 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 210019 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 210019 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 210022 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 210023 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 210024 INFO  (zkCallback-119-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@78bc98b name:ZooKeeperConnection Watcher:127.0.0.1:53816/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 210024 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 210024 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 210025 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 210025 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 210029 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 210029 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 210032 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 210033 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 210036 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 210036 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 210039 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 210040 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 210043 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 210043 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 210046 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 210047 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 210050 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 210050 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 210053 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 210054 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 210056 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 210056 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 210059 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 210059 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 210062 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 210062 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 210401 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DDD71E7F89D6D30C-001/control-001/cores/collection1
   [junit4]   2> 210422 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 210433 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@c21a9b0{/_u,null,AVAILABLE}
   [junit4]   2> 210559 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.e.j.s.ServerConnector Started ServerConnector@7e05b4cd{SSL-http/1.1}{127.0.0.1:५३१४०}
   [junit4]   2> 210559 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.e.j.s.Server Started @२१३६९६ms
   [junit4]   2> 210559 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/_u, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DDD71E7F89D6D30C-001/control-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DDD71E7F89D6D30C-001/tempDir-001/control/data, hostPort=53140}
   [junit4]   2> 210560 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@265f00f9
   [junit4]   2> 210560 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DDD71E7F89D6D30C-001/control-001'
   [junit4]   2> 210560 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 210560 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 210560 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 210561 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 210564 INFO  (zkCallback-120-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@49ef2d0 name:ZooKeeperConnection Watcher:127.0.0.1:53816/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 210564 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 210564 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 210567 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 210567 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DDD71E7F89D6D30C-001/control-001/solr.xml
   [junit4]   2> 210590 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DDD71E7F89D6D30C-001/control-001/cores
   [junit4]   2> 210591 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.CoreContainer New CoreContainer 690439547
   [junit4]   2> 210591 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DDD71E7F89D6D30C-001/control-001]
   [junit4]   2> 210591 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DDD71E7F89D6D30C-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DDD71E7F89D6D30C-001/control-001/lib
   [junit4]   2> 210591 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 210604 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 210606 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 210607 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 210607 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53816/solr
   [junit4]   2> 210607 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 210607 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 210609 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 210610 INFO  (zkCallback-122-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6fb6dacf name:ZooKeeperConnection Watcher:127.0.0.1:53816 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 210611 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 210611 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 210614 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:53140__u    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 210616 INFO  (zkCallback-123-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@64a8fa0a name:ZooKeeperConnection Watcher:127.0.0.1:53816/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 210616 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:53140__u    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 210618 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:53140__u    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 210621 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:53140__u    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 210625 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:53140__u    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 210628 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:53140__u    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 210631 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:53140__u    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 210635 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:53140__u    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 210637 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:53140__u    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 210639 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:53140__u    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 210640 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:53140__u    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 210642 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:53140__u    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53140__u
   [junit4]   2> 210642 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:53140__u    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:53140__u
   [junit4]   2> 210645 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:53140__u    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 210646 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:53140__u    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 210649 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:53140__u    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 210650 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:53140__u    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95088328195899399-127.0.0.1:53140__u-n_0000000000
   [junit4]   2> 210650 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:53140__u    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:53140__u
   [junit4]   2> 210651 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:53140__u    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 210653 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:53140__u    ] o.a.s.c.Overseer Overseer (id=95088328195899399-127.0.0.1:53140__u-n_0000000000) starting
   [junit4]   2> 210656 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:53140__u    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 210665 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:53140__u    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 210666 INFO  (OverseerCollectionConfigSetProcessor-95088328195899399-127.0.0.1:53140__u-n_0000000000) [n:127.0.0.1:53140__u    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 210666 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:53140__u    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 210667 INFO  (OverseerStateUpdate-95088328195899399-127.0.0.1:53140__u-n_0000000000) [n:127.0.0.1:53140__u    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 210695 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:53140__u    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 210695 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:53140__u    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 210696 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:53140__u    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DDD71E7F89D6D30C-001/control-001/cores
   [junit4]   2> 210697 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:53140__u    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {loadOnStartup=true, collection=control_collection, schema=schema.xml, transient=false, shard=, name=collection1, configSetProperties=configsetprops.json, coreNodeName=, dataDir=data/, config=solrconfig.xml}
   [junit4]   2> 210697 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:53140__u    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DDD71E7F89D6D30C-001/control-001/cores/collection1
   [junit4]   2> 210697 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:53140__u    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 210698 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:53140__u    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 210698 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:53140__u    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 210699 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 210699 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 210702 INFO  (OverseerStateUpdate-95088328195899399-127.0.0.1:53140__u-n_0000000000) [n:127.0.0.1:53140__u    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"https://127.0.0.1:53140/_u",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:53140__u",
   [junit4]   2>   "state":"down"} current state version: 0
   [junit4]   2> 210703 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 210704 INFO  (OverseerStateUpdate-95088328195899399-127.0.0.1:53140__u-n_0000000000) [n:127.0.0.1:53140__u    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"https://127.0.0.1:53140/_u",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:53140__u",
   [junit4]   2>   "state":"down"}
   [junit4]   2> 210705 INFO  (OverseerStateUpdate-95088328195899399-127.0.0.1:53140__u-n_0000000000) [n:127.0.0.1:53140__u    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 210705 INFO  (OverseerStateUpdate-95088328195899399-127.0.0.1:53140__u-n_0000000000) [n:127.0.0.1:53140__u    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 210708 INFO  (zkCallback-123-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 211704 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 211704 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 211705 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 211705 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 211706 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 211706 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DDD71E7F89D6D30C-001/control-001/cores/collection1'
   [junit4]   2> 211706 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 211706 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 211709 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 211719 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 211742 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 211778 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 211787 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 212055 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 212064 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 212066 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 212081 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 212084 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 212087 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 212088 WARN  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 212088 WARN  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 212088 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 212089 WARN  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 212089 WARN  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 212090 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 212090 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 212090 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 212090 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DDD71E7F89D6D30C-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DDD71E7F89D6D30C-001/control-001/cores/collection1/data/]
   [junit4]   2> 212090 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6735b09d
   [junit4]   2> 212091 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DDD71E7F89D6D30C-001/control-001/cores/collection1/data
   [junit4]   2> 212091 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DDD71E7F89D6D30C-001/control-001/cores/collection1/data/index/
   [junit4]   2> 212091 WARN  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DDD71E7F89D6D30C-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 212092 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DDD71E7F89D6D30C-001/control-001/cores/collection1/data/index
   [junit4]   2> 212092 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=2, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 212093 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@660294f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@133804ce; maxCacheMB=0.7421875 maxMergeSizeMB=0.6630859375)),segFN=segments_1,generation=1}
   [junit4]   2> 212093 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 212093 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DDD71E7F89D6D30C-001/control-001/cores/collection1/data/
   [junit4]   2> 212094 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DDD71E7F89D6D30C-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 212100 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 212100 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 212100 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 212100 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 212101 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 212101 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 212101 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 212101 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 212102 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 212102 WARN  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 212105 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 212106 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 212107 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 212108 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 212113 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/ping,/admin/plugins,/config,/admin/properties,/schema,/update/json,/update/json/docs,/replication,/update,/update/csv,/dump,/admin/file,/admin/mbeans,/admin/system,/get,/admin/segments,/admin/luke,standard,/admin/threads,/admin/logging
   [junit4]   2> 212114 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 212115 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 212115 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 212115 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 212115 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 212116 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=12, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=78.923828125, floorSegmentMB=0.265625, forceMergeDeletesPctAllowed=22.14482694385272, segmentsPerTier=20.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6279470937490147
   [junit4]   2> 212116 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@660294f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@133804ce; maxCacheMB=0.7421875 maxMergeSizeMB=0.6630859375)),segFN=segments_1,generation=1}
   [junit4]   2> 212116 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 212117 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@11e5b5de[collection1] main
   [junit4]   2> 212117 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 212118 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 212118 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 212118 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 212118 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 212118 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 212119 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 212119 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 212119 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 212119 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 212120 INFO  (searcherExecutor-495-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:53140__u s:shard1 r:core_node1) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@11e5b5de[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 212120 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 212120 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 212120 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 212120 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1521413253505220608
   [junit4]   2> 212122 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1521413253505220608
   [junit4]   2> 212122 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 212123 INFO  (coreLoadExecutor-494-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 212123 INFO  (coreZkRegister-488-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:53140__u s:shard1 r:core_node1) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:https://127.0.0.1:53140/_u collection:control_collection shard:shard1
   [junit4]   2> 212124 INFO  (coreZkRegister-488-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:53140__u s:shard1 r:core_node1) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 212128 INFO  (coreZkRegister-488-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:53140__u s:shard1 r:core_node1) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95088328195899399-core_node1-n_0000000000
   [junit4]   2> 212128 INFO  (coreZkRegister-488-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:53140__u s:shard1 r:core_node1) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=180000
   [junit4]   2> 212130 INFO  (coreZkRegister-488-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:53140__u s:shard1 r:core_node1) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 212131 INFO  (coreZkRegister-488-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:53140__u s:shard1 r:core_node1) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 212131 INFO  (coreZkRegister-488-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:53140__u s:shard1 r:core_node1) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:53140/_u/collection1/
   [junit4]   2> 212131 INFO  (coreZkRegister-488-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:53140__u s:shard1 r:core_node1) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 212131 INFO  (coreZkRegister-488-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:53140__u s:shard1 r:core_node1) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:53140/_u/collection1/ has no replicas
   [junit4]   2> 212131 INFO  (OverseerStateUpdate-95088328195899399-127.0.0.1:53140__u-n_0000000000) [n:127.0.0.1:53140__u    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 212132 INFO  (coreZkRegister-488-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:53140__u s:shard1 r:core_node1) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 212134 INFO  (coreZkRegister-488-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:53140__u s:shard1 r:core_node1) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 212136 INFO  (coreZkRegister-488-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:53140__u s:shard1 r:core_node1) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:53140/_u/collection1/ shard1
   [junit4]   2> 212137 INFO  (OverseerStateUpdate-95088328195899399-127.0.0.1:53140__u-n_0000000000) [n:127.0.0.1:53140__u    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"https://127.0.0.1:53140/_u",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 212204 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 212205 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 212206 INFO  (zkCallback-125-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3b233d0d name:ZooKeeperConnection Watcher:127.0.0.1:53816/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 212207 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 212207 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 212207 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 212210 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 212210 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 212210 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 212211 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 212212 INFO  (zkCallback-126-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7917e64c name:ZooKeeperConnection Watcher:127.0.0.1:53816/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 212212 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 212212 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 212216 INFO  (OverseerStateUpdate-95088328195899399-127.0.0.1:53140__u-n_0000000000) [n:127.0.0.1:53140__u    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 1
   [junit4]   2> 212216 INFO  (OverseerStateUpdate-95088328195899399-127.0.0.1:53140__u-n_0000000000) [n:127.0.0.1:53140__u    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 212220 INFO  (zkCallback-123-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 212220 INFO  (zkCallback-125-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 212237 INFO  (coreZkRegister-488-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:53140__u s:shard1 r:core_node1) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are https://127.0.0.1:53140/_u/collection1/ and leader is https://127.0.0.1:53140/_u/collection1/
   [junit4]   2> 212238 INFO  (coreZkRegister-488-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:53140__u s:shard1 r:core_node1) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:53140/_u
   [junit4]   2> 212238 INFO  (coreZkRegister-488-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:53140__u s:shard1 r:core_node1) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 212238 INFO  (coreZkRegister-488-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:53140__u s:shard1 r:core_node1) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 212238 INFO  (coreZkRegister-488-thread-1-processing-x:collection1 c:control_collection n:127.0.0.1:53140__u s:shard1 r:core_node1) [n:127.0.0.1:53140__u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 212240 INFO  (OverseerStateUpdate-95088328195899399-127.0.0.1:53140__u-n_0000000000) [n:127.0.0.1:53140__u    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "base_url":"https://127.0.0.1:53140/_u",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "node_name":"127.0.0.1:53140__u",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 212241 INFO  (OverseerStateUpdate-95088328195899399-127.0.0.1:53140__u-n_0000000000) [n:127.0.0.1:53140__u    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "base_url":"https://127.0.0.1:53140/_u",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "node_name":"127.0.0.1:53140__u",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 212241 INFO  (OverseerStateUpdate-95088328195899399-127.0.0.1:53140__u-n_0000000000) [n:127.0.0.1:53140__u    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 212346 INFO  (zkCallback-125-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 212346 INFO  (zkCallback-123-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 212502 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DDD71E7F89D6D30C-001/shard-1-001/cores/collection1
   [junit4]   2> 212503 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DDD71E7F89D6D30C-001/shard-1-001
   [junit4]   2> 212504 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 212506 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2d6e894c{/_u,null,AVAILABLE}
   [junit4]   2> 212518 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.e.j.s.ServerConnector Started ServerConnector@33f7f3e1{SSL-http/1.1}{127.0.0.1:६०५४७}
   [junit4]   2> 212518 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.e.j.s.Server Started @२१५६५५ms
   [junit4]   2> 212518 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=60547, hostContext=/_u, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DDD71E7F89D6D30C-001/shard-1-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DDD71E7F89D6D30C-001/tempDir-001/jetty1, solrconfig=solrconfig.xml}
   [junit4]   2> 212518 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@265f00f9
   [junit4]   2> 212519 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DDD71E7F89D6D30C-001/shard-1-001'
   [junit4]   2> 212519 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 212519 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 212519 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 212520 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 212521 INFO  (zkCallback-127-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@47141a98 name:ZooKeeperConnection Watcher:127.0.0.1:53816/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 212521 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 212521 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 212523 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 212523 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DDD71E7F89D6D30C-001/shard-1-001/solr.xml
   [junit4]   2> 212537 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DDD71E7F89D6D30C-001/shard-1-001/cores
   [junit4]   2> 212537 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.CoreContainer New CoreContainer 219452949
   [junit4]   2> 212537 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DDD71E7F89D6D30C-001/shard-1-001]
   [junit4]   2> 212537 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DDD71E7F89D6D30C-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DDD71E7F89D6D30C-001/shard-1-001/lib
   [junit4]   2> 212537 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 212541 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 212543 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 212543 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 212543 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53816/solr
   [junit4]   2> 212543 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 212543 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 212544 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 212545 INFO  (zkCallback-129-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@297361e name:ZooKeeperConnection Watcher:127.0.0.1:53816 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 212546 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 212546 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 212548 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:60547__u    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 212549 INFO  (zkCallback-130-thread-1-processing-n:127.0.0.1:60547__u) [n:127.0.0.1:60547__u    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@65f5c1f0 name:ZooKeeperConnection Watcher:127.0.0.1:53816/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 212549 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:60547__u    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 212556 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:60547__u    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 213564 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:60547__u    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60547__u
   [junit4]   2> 213565 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:60547__u    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:60547__u
   [junit4]   2> 213569 INFO  (zkCallback-123-thread-1-processing-n:127.0.0.1:53140__u) [n:127.0.0.1:53140__u    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 213569 INFO  (zkCallback-130-thread-1-processing-n:127.0.0.1:60547__u) [n:127.0.0.1:60547__u    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 213569 INFO  (zkCallback-125-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 213574 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:60547__u    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 213575 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:60547__u    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95088328195899404-127.0.0.1:60547__u-n_0000000001
   [junit4]   2> 213576 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:60547__u    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/95088328195899399-127.0.0.1:53140__u-n_0000000000 to know if I could be the leader
   [junit4]   2> 213591 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:60547__u    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 213591 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:60547__u    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 213592 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:60547__u    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_

[...truncated too long message...]

ests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CloudExitableDirectoryReaderTest_DDD71E7F89D6D30C-001/shard-4-001/cores/collection1/data;done=false>>]
   [junit4]   2> 741988 INFO  (zkCallback-531-thread-2-processing-n:127.0.0.1:59444_phpp) [n:127.0.0.1:59444_phpp c:collection1 s:shard2 r:core_node4 x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CloudExitableDirectoryReaderTest_DDD71E7F89D6D30C-001/shard-4-001/cores/collection1/data
   [junit4]   2> 741989 INFO  (TEST-CloudExitableDirectoryReaderTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@324f6e41{HTTP/1.1}{127.0.0.1:0}
   [junit4]   2> 741989 INFO  (TEST-CloudExitableDirectoryReaderTest.test-seed#[DDD71E7F89D6D30C]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@245e77e5{/phpp,null,UNAVAILABLE}
   [junit4]   2> 741991 INFO  (TEST-CloudExitableDirectoryReaderTest.test-seed#[DDD71E7F89D6D30C]) [n:127.0.0.1:59444_phpp c:collection1 s:shard2 r:core_node4 x:collection1] o.a.s.c.ZkTestServer connecting to 127.0.0.1:41571 41571
   [junit4]   2> 742171 INFO  (Thread-6547) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:41571 41571
   [junit4]   2> 742172 WARN  (Thread-6547) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	7	/solr/aliases.json
   [junit4]   2> 	6	/solr/security.json
   [junit4]   2> 	5	/solr/configs/conf1
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	11	/solr/clusterstate.json
   [junit4]   2> 	4	/solr/collections/collection1/state.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	11	/solr/live_nodes
   [junit4]   2> 	7	/solr/collections
   [junit4]   2> 	5	/solr/overseer/queue
   [junit4]   2> 	5	/solr/overseer/collection-queue-work
   [junit4]   2> 	3	/solr/overseer/queue-work
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=CloudExitableDirectoryReaderTest -Dtests.method=test -Dtests.seed=DDD71E7F89D6D30C -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=ru_RU -Dtests.timezone=GMT -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   27.4s J0 | CloudExitableDirectoryReaderTest.test <<<
   [junit4]    > Throwable #1: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:47131/phpp/collection1]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([DDD71E7F89D6D30C:558321A5272ABEF4]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.queryServer(AbstractFullDistribZkTestBase.java:1378)
   [junit4]    > 	at org.apache.solr.cloud.CloudExitableDirectoryReaderTest.assertPartialResults(CloudExitableDirectoryReaderTest.java:101)
   [junit4]    > 	at org.apache.solr.cloud.CloudExitableDirectoryReaderTest.doTimeoutTests(CloudExitableDirectoryReaderTest.java:73)
   [junit4]    > 	at org.apache.solr.cloud.CloudExitableDirectoryReaderTest.test(CloudExitableDirectoryReaderTest.java:52)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:965)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:940)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:47131/phpp/collection1: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	... 51 more
   [junit4]   2> 742176 INFO  (SUITE-CloudExitableDirectoryReaderTest-seed#[DDD71E7F89D6D30C]-worker) [n:127.0.0.1:59444_phpp c:collection1 s:shard2 r:core_node4 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CloudExitableDirectoryReaderTest_DDD71E7F89D6D30C-001
   [junit4]   2> Dec 24, 2015 5:01:14 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {range_facet_l=PostingsFormat(name=LuceneFixedGap), _version_=PostingsFormat(name=LuceneFixedGap), range_facet_i_dv=PostingsFormat(name=LuceneFixedGap), name=BlockTreeOrds(blocksize=128), multiDefault=PostingsFormat(name=Direct), id=BlockTreeOrds(blocksize=128), timestamp=PostingsFormat(name=LuceneFixedGap), range_facet_l_dv=BlockTreeOrds(blocksize=128), intDefault=PostingsFormat(name=LuceneFixedGap)}, docValues:{range_facet_i_dv=DocValuesFormat(name=Asserting), timestamp=DocValuesFormat(name=Asserting), range_facet_l_dv=DocValuesFormat(name=Lucene54)}, sim=DefaultSimilarity, locale=ru_RU, timezone=GMT
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=2,free=161618064,total=406847488
   [junit4]   2> NOTE: All tests run in this JVM: [OverseerCollectionConfigSetProcessorTest, TestOmitPositions, HdfsWriteToMultipleCollectionsTest, TestFieldTypeCollectionResource, TestIBSimilarityFactory, DateRangeFieldTest, TestLuceneMatchVersion, CurrencyFieldOpenExchangeTest, XsltUpdateRequestHandlerTest, TestStressLucene, TestBlendedInfixSuggestions, BitVectorTest, ConfigSetsAPITest, TestDocBasedVersionConstraints, DeleteInactiveReplicaTest, AsyncMigrateRouteKeyTest, TestRawResponseWriter, SolrCoreTest, TestArbitraryIndexDir, AlternateDirectoryTest, PeerSyncTest, ZkNodePropsTest, StandardRequestHandlerTest, SyncSliceTest, HLLSerializationTest, DistributedQueryElevationComponentTest, AnalysisAfterCoreReloadTest, SynonymTokenizerTest, DistributedDebugComponentTest, TestPhraseSuggestions, BasicAuthIntegrationTest, VersionInfoTest, TestAnalyzedSuggestions, TestFaceting, EchoParamsTest, HdfsSyncSliceTest, TestLMDirichletSimilarityFactory, TestCrossCoreJoin, OverseerTaskQueueTest, SuggestComponentContextFilterQueryTest, TestMiniSolrCloudClusterSSL, PathHierarchyTokenizerFactoryTest, TestLazyCores, TestReplicaProperties, OutputWriterTest, TestSolrCLIRunExample, TestSolr4Spatial, DocumentAnalysisRequestHandlerTest, TestFoldingMultitermQuery, TestManagedSchema, TestPseudoReturnFields, DistribJoinFromCollectionTest, TestExactSharedStatsCache, TestRestoreCore, TestClassicSimilarityFactory, CoreAdminCreateDiscoverTest, TestSchemaVersionResource, CloudExitableDirectoryReaderTest]
   [junit4] Completed [103/555 (2!)] on J0 in 27.93s, 1 test, 1 error <<< FAILURES!

[...truncated 1459 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1456: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1010: There were test failures: 555 suites (7 ignored), 2209 tests, 2 errors, 33 ignored (18 assumptions) [seed: DDD71E7F89D6D30C]

Total time: 195 minutes 28 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1054 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1054/

3 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at https://127.0.0.1:34590/m_mn/w: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 500 </title> </head> <body> <h2>HTTP ERROR: 500</h2> <p>Problem accessing /m_mn/w/admin/cores. Reason: <pre>    {msg=Java heap space,trace=java.lang.OutOfMemoryError: Java heap space ,code=500}</pre></p> <hr /><i><small>Powered by Jetty://</small></i> </body> </html> 

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:34590/m_mn/w: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /m_mn/w/admin/cores. Reason:
<pre>    {msg=Java heap space,trace=java.lang.OutOfMemoryError: Java heap space
,code=500}</pre></p>
<hr /><i><small>Powered by Jetty://</small></i>
</body>
</html>

	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:543)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:295)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:412)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:162)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:965)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:940)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=78190, name=collection3, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=78190, name=collection3, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:59673: collection already exists: awholynewstresscollection_collection3_2
	at __randomizedtesting.SeedInfo.seed([D9B0A5512A1C7560]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:881)


FAILED:  org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload

Error Message:
expected:<[{indexVersion=1450828426231,generation=2,filelist=[_la.fdt, _la.fdx, _la.fnm, _la.nvd, _la.nvm, _la.si, _la_LuceneFixedGap_0.doc, _la_LuceneFixedGap_0.tib, _la_LuceneFixedGap_0.tii, _nx.fdt, _nx.fdx, _nx.fnm, _nx.nvd, _nx.nvm, _nx.si, _nx_LuceneFixedGap_0.doc, _nx_LuceneFixedGap_0.tib, _nx_LuceneFixedGap_0.tii, _p8.fdt, _p8.fdx, _p8.fnm, _p8.nvd, _p8.nvm, _p8.si, _p8_LuceneFixedGap_0.doc, _p8_LuceneFixedGap_0.tib, _p8_LuceneFixedGap_0.tii, _pj.fdt, _pj.fdx, _pj.fnm, _pj.nvd, _pj.nvm, _pj.si, _pj_LuceneFixedGap_0.doc, _pj_LuceneFixedGap_0.tib, _pj_LuceneFixedGap_0.tii, _po.fdt, _po.fdx, _po.fnm, _po.nvd, _po.nvm, _po.si, _po_LuceneFixedGap_0.doc, _po_LuceneFixedGap_0.tib, _po_LuceneFixedGap_0.tii, _pp.fdt, _pp.fdx, _pp.fnm, _pp.nvd, _pp.nvm, _pp.si, _pp_LuceneFixedGap_0.doc, _pp_LuceneFixedGap_0.tib, _pp_LuceneFixedGap_0.tii, _pq.fdt, _pq.fdx, _pq.fnm, _pq.nvd, _pq.nvm, _pq.si, _pq_LuceneFixedGap_0.doc, _pq_LuceneFixedGap_0.tib, _pq_LuceneFixedGap_0.tii, segments_2]}]> but was:<[{indexVersion=1450828426231,generation=3,filelist=[_la.fdt, _la.fdx, _la.fnm, _la.nvd, _la.nvm, _la.si, _la_LuceneFixedGap_0.doc, _la_LuceneFixedGap_0.tib, _la_LuceneFixedGap_0.tii, _nx.fdt, _nx.fdx, _nx.fnm, _nx.nvd, _nx.nvm, _nx.si, _nx_LuceneFixedGap_0.doc, _nx_LuceneFixedGap_0.tib, _nx_LuceneFixedGap_0.tii, _p8.fdt, _p8.fdx, _p8.fnm, _p8.nvd, _p8.nvm, _p8.si, _p8_LuceneFixedGap_0.doc, _p8_LuceneFixedGap_0.tib, _p8_LuceneFixedGap_0.tii, _pj.fdt, _pj.fdx, _pj.fnm, _pj.nvd, _pj.nvm, _pj.si, _pj_LuceneFixedGap_0.doc, _pj_LuceneFixedGap_0.tib, _pj_LuceneFixedGap_0.tii, _po.fdt, _po.fdx, _po.fnm, _po.nvd, _po.nvm, _po.si, _po_LuceneFixedGap_0.doc, _po_LuceneFixedGap_0.tib, _po_LuceneFixedGap_0.tii, _pr.fdt, _pr.fdx, _pr.fnm, _pr.nvd, _pr.nvm, _pr.si, _pr_LuceneFixedGap_0.doc, _pr_LuceneFixedGap_0.tib, _pr_LuceneFixedGap_0.tii, segments_3]}, {indexVersion=1450828426231,generation=2,filelist=[_la.fdt, _la.fdx, _la.fnm, _la.nvd, _la.nvm, _la.si, _la_LuceneFixedGap_0.doc, _la_LuceneFixedGap_0.tib, _la_LuceneFixedGap_0.tii, _nx.fdt, _nx.fdx, _nx.fnm, _nx.nvd, _nx.nvm, _nx.si, _nx_LuceneFixedGap_0.doc, _nx_LuceneFixedGap_0.tib, _nx_LuceneFixedGap_0.tii, _p8.fdt, _p8.fdx, _p8.fnm, _p8.nvd, _p8.nvm, _p8.si, _p8_LuceneFixedGap_0.doc, _p8_LuceneFixedGap_0.tib, _p8_LuceneFixedGap_0.tii, _pj.fdt, _pj.fdx, _pj.fnm, _pj.nvd, _pj.nvm, _pj.si, _pj_LuceneFixedGap_0.doc, _pj_LuceneFixedGap_0.tib, _pj_LuceneFixedGap_0.tii, _po.fdt, _po.fdx, _po.fnm, _po.nvd, _po.nvm, _po.si, _po_LuceneFixedGap_0.doc, _po_LuceneFixedGap_0.tib, _po_LuceneFixedGap_0.tii, _pp.fdt, _pp.fdx, _pp.fnm, _pp.nvd, _pp.nvm, _pp.si, _pp_LuceneFixedGap_0.doc, _pp_LuceneFixedGap_0.tib, _pp_LuceneFixedGap_0.tii, _pq.fdt, _pq.fdx, _pq.fnm, _pq.nvd, _pq.nvm, _pq.si, _pq_LuceneFixedGap_0.doc, _pq_LuceneFixedGap_0.tib, _pq_LuceneFixedGap_0.tii, segments_2]}]>

Stack Trace:
java.lang.AssertionError: expected:<[{indexVersion=1450828426231,generation=2,filelist=[_la.fdt, _la.fdx, _la.fnm, _la.nvd, _la.nvm, _la.si, _la_LuceneFixedGap_0.doc, _la_LuceneFixedGap_0.tib, _la_LuceneFixedGap_0.tii, _nx.fdt, _nx.fdx, _nx.fnm, _nx.nvd, _nx.nvm, _nx.si, _nx_LuceneFixedGap_0.doc, _nx_LuceneFixedGap_0.tib, _nx_LuceneFixedGap_0.tii, _p8.fdt, _p8.fdx, _p8.fnm, _p8.nvd, _p8.nvm, _p8.si, _p8_LuceneFixedGap_0.doc, _p8_LuceneFixedGap_0.tib, _p8_LuceneFixedGap_0.tii, _pj.fdt, _pj.fdx, _pj.fnm, _pj.nvd, _pj.nvm, _pj.si, _pj_LuceneFixedGap_0.doc, _pj_LuceneFixedGap_0.tib, _pj_LuceneFixedGap_0.tii, _po.fdt, _po.fdx, _po.fnm, _po.nvd, _po.nvm, _po.si, _po_LuceneFixedGap_0.doc, _po_LuceneFixedGap_0.tib, _po_LuceneFixedGap_0.tii, _pp.fdt, _pp.fdx, _pp.fnm, _pp.nvd, _pp.nvm, _pp.si, _pp_LuceneFixedGap_0.doc, _pp_LuceneFixedGap_0.tib, _pp_LuceneFixedGap_0.tii, _pq.fdt, _pq.fdx, _pq.fnm, _pq.nvd, _pq.nvm, _pq.si, _pq_LuceneFixedGap_0.doc, _pq_LuceneFixedGap_0.tib, _pq_LuceneFixedGap_0.tii, segments_2]}]> but was:<[{indexVersion=1450828426231,generation=3,filelist=[_la.fdt, _la.fdx, _la.fnm, _la.nvd, _la.nvm, _la.si, _la_LuceneFixedGap_0.doc, _la_LuceneFixedGap_0.tib, _la_LuceneFixedGap_0.tii, _nx.fdt, _nx.fdx, _nx.fnm, _nx.nvd, _nx.nvm, _nx.si, _nx_LuceneFixedGap_0.doc, _nx_LuceneFixedGap_0.tib, _nx_LuceneFixedGap_0.tii, _p8.fdt, _p8.fdx, _p8.fnm, _p8.nvd, _p8.nvm, _p8.si, _p8_LuceneFixedGap_0.doc, _p8_LuceneFixedGap_0.tib, _p8_LuceneFixedGap_0.tii, _pj.fdt, _pj.fdx, _pj.fnm, _pj.nvd, _pj.nvm, _pj.si, _pj_LuceneFixedGap_0.doc, _pj_LuceneFixedGap_0.tib, _pj_LuceneFixedGap_0.tii, _po.fdt, _po.fdx, _po.fnm, _po.nvd, _po.nvm, _po.si, _po_LuceneFixedGap_0.doc, _po_LuceneFixedGap_0.tib, _po_LuceneFixedGap_0.tii, _pr.fdt, _pr.fdx, _pr.fnm, _pr.nvd, _pr.nvm, _pr.si, _pr_LuceneFixedGap_0.doc, _pr_LuceneFixedGap_0.tib, _pr_LuceneFixedGap_0.tii, segments_3]}, {indexVersion=1450828426231,generation=2,filelist=[_la.fdt, _la.fdx, _la.fnm, _la.nvd, _la.nvm, _la.si, _la_LuceneFixedGap_0.doc, _la_LuceneFixedGap_0.tib, _la_LuceneFixedGap_0.tii, _nx.fdt, _nx.fdx, _nx.fnm, _nx.nvd, _nx.nvm, _nx.si, _nx_LuceneFixedGap_0.doc, _nx_LuceneFixedGap_0.tib, _nx_LuceneFixedGap_0.tii, _p8.fdt, _p8.fdx, _p8.fnm, _p8.nvd, _p8.nvm, _p8.si, _p8_LuceneFixedGap_0.doc, _p8_LuceneFixedGap_0.tib, _p8_LuceneFixedGap_0.tii, _pj.fdt, _pj.fdx, _pj.fnm, _pj.nvd, _pj.nvm, _pj.si, _pj_LuceneFixedGap_0.doc, _pj_LuceneFixedGap_0.tib, _pj_LuceneFixedGap_0.tii, _po.fdt, _po.fdx, _po.fnm, _po.nvd, _po.nvm, _po.si, _po_LuceneFixedGap_0.doc, _po_LuceneFixedGap_0.tib, _po_LuceneFixedGap_0.tii, _pp.fdt, _pp.fdx, _pp.fnm, _pp.nvd, _pp.nvm, _pp.si, _pp_LuceneFixedGap_0.doc, _pp_LuceneFixedGap_0.tib, _pp_LuceneFixedGap_0.tii, _pq.fdt, _pq.fdx, _pq.fnm, _pq.nvd, _pq.nvm, _pq.si, _pq_LuceneFixedGap_0.doc, _pq_LuceneFixedGap_0.tib, _pq_LuceneFixedGap_0.tii, segments_2]}]>
	at __randomizedtesting.SeedInfo.seed([D9B0A5512A1C7560:FC67BE615A547B63]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:147)
	at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1140)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11069 lines...]
   [junit4] Suite: org.apache.solr.handler.TestReplicationHandler
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/init-core-data-001
   [junit4]   2> 2005126 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.SolrTestCaseJ4 ###Starting doTestStopPoll
   [junit4]   2> 2005127 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-001/collection1
   [junit4]   2> 2005147 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 2005173 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@a1677e5{/solr,null,AVAILABLE}
   [junit4]   2> 2005176 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.e.j.s.ServerConnector Started ServerConnector@4919608f{HTTP/1.1}{127.0.0.1:53319}
   [junit4]   2> 2005176 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.e.j.s.Server Started @2008179ms
   [junit4]   2> 2005176 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-001/collection1/data, hostPort=53319}
   [junit4]   2> 2005176 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 2005177 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-001'
   [junit4]   2> 2005177 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2005177 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2005177 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-001/solr.xml
   [junit4]   2> 2005195 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-001/.
   [junit4]   2> 2005195 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.c.CoreContainer New CoreContainer 1088432092
   [junit4]   2> 2005195 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-001]
   [junit4]   2> 2005195 WARN  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-001/lib
   [junit4]   2> 2005195 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2005196 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2005197 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2005197 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2005197 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2005197 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2005198 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-001/.
   [junit4]   2> 2005200 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {config=solrconfig.xml, configSetProperties=configsetprops.json, schema=schema.xml, loadOnStartup=true, name=collection1, transient=false, dataDir=data/}
   [junit4]   2> 2005200 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-001/./collection1
   [junit4]   2> 2005200 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 2005200 INFO  (coreLoadExecutor-5032-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-001/collection1'
   [junit4]   2> 2005200 INFO  (coreLoadExecutor-5032-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2005200 INFO  (coreLoadExecutor-5032-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2005205 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 2005205 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2005209 INFO  (coreLoadExecutor-5032-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2005227 INFO  (coreLoadExecutor-5032-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 2005254 INFO  (coreLoadExecutor-5032-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2005262 INFO  (coreLoadExecutor-5032-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2005273 INFO  (coreLoadExecutor-5032-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2005276 INFO  (coreLoadExecutor-5032-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-001/collection1'
   [junit4]   2> 2005276 INFO  (coreLoadExecutor-5032-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-001/./collection1
   [junit4]   2> 2005276 INFO  (coreLoadExecutor-5032-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2005276 INFO  (coreLoadExecutor-5032-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-001/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-001/./collection1/data/]
   [junit4]   2> 2005276 INFO  (coreLoadExecutor-5032-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 2005276 INFO  (coreLoadExecutor-5032-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-001/./collection1/data
   [junit4]   2> 2005277 INFO  (coreLoadExecutor-5032-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-001/./collection1/data/index/
   [junit4]   2> 2005277 WARN  (coreLoadExecutor-5032-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-001/./collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2005277 INFO  (coreLoadExecutor-5032-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-001/./collection1/data/index
   [junit4]   2> 2005277 INFO  (coreLoadExecutor-5032-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=22, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8917305865821082]
   [junit4]   2> 2005277 INFO  (coreLoadExecutor-5032-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@6da8124a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3548f084),segFN=segments_1,generation=1}
   [junit4]   2> 2005278 INFO  (coreLoadExecutor-5032-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2005278 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-001/./collection1/data/
   [junit4]   2> 2005279 INFO  (coreLoadExecutor-5032-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2005283 INFO  (coreLoadExecutor-5032-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2005284 INFO  (coreLoadExecutor-5032-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2005285 INFO  (coreLoadExecutor-5032-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2005286 INFO  (coreLoadExecutor-5032-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2005292 INFO  (coreLoadExecutor-5032-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 2005292 INFO  (coreLoadExecutor-5032-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /schema,/admin/ping,/admin/system,/update,/update/json/docs,/config,/admin/mbeans,lazy,/admin/properties,/admin/plugins,/admin/file,/update/csv,/admin/threads,/admin/luke,/admin/segments,defaults,/get,/admin/logging,standard,/update/json,/replication
   [junit4]   2> 2005293 INFO  (coreLoadExecutor-5032-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2005294 INFO  (coreLoadExecutor-5032-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2005294 INFO  (coreLoadExecutor-5032-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2005294 INFO  (coreLoadExecutor-5032-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=2, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 2005294 INFO  (coreLoadExecutor-5032-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@6da8124a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3548f084),segFN=segments_1,generation=1}
   [junit4]   2> 2005294 INFO  (coreLoadExecutor-5032-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2005294 INFO  (coreLoadExecutor-5032-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@79ac6df[collection1] main
   [junit4]   2> 2005295 INFO  (coreLoadExecutor-5032-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-001/collection1/conf
   [junit4]   2> 2005295 INFO  (coreLoadExecutor-5032-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-001/collection1/conf}
   [junit4]   2> 2005295 INFO  (coreLoadExecutor-5032-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-001/collection1/conf
   [junit4]   2> 2005295 INFO  (coreLoadExecutor-5032-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2005295 INFO  (coreLoadExecutor-5032-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml,xslt/dummy.xsl
   [junit4]   2> 2005295 INFO  (coreLoadExecutor-5032-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2005296 INFO  (coreLoadExecutor-5032-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2005296 INFO  (searcherExecutor-5033-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@79ac6df[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2005707 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-002/collection1
   [junit4]   2> 2005723 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 2005741 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@62acfd7a{/solr,null,AVAILABLE}
   [junit4]   2> 2005741 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.e.j.s.ServerConnector Started ServerConnector@4b358732{HTTP/1.1}{127.0.0.1:58243}
   [junit4]   2> 2005741 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.e.j.s.Server Started @2008744ms
   [junit4]   2> 2005741 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-002/collection1/data, hostContext=/solr, hostPort=58243}
   [junit4]   2> 2005742 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 2005742 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-002'
   [junit4]   2> 2005742 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2005742 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2005742 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-002/solr.xml
   [junit4]   2> 2005763 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-002/.
   [junit4]   2> 2005763 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.c.CoreContainer New CoreContainer 60474042
   [junit4]   2> 2005763 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-002]
   [junit4]   2> 2005763 WARN  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-002/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-002/lib
   [junit4]   2> 2005764 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2005765 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2005765 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2005770 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2005770 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2005770 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2005771 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-002/.
   [junit4]   2> 2005772 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {transient=false, schema=schema.xml, loadOnStartup=true, dataDir=data/, name=collection1, configSetProperties=configsetprops.json, config=solrconfig.xml}
   [junit4]   2> 2005772 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-002/./collection1
   [junit4]   2> 2005772 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 2005777 INFO  (coreLoadExecutor-5042-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-002/collection1'
   [junit4]   2> 2005778 INFO  (coreLoadExecutor-5042-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2005778 INFO  (coreLoadExecutor-5042-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2005790 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 2005790 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[D9B0A5512A1C7560]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2005790 INFO  (coreLoadExecutor-5042-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2005821 INFO  (coreLoadExecutor-5042-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 2005931 INFO  (coreLoadExecutor-5042-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2005935 INFO  (coreLoadExecutor-5042-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2005943 INFO  (coreLoadExecutor-5042-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2005945 INFO  (coreLoadExecutor-5042-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-002/collection1'
   [junit4]   2> 2005945 INFO  (coreLoadExecutor-5042-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-002/./collection1
   [junit4]   2> 2005945 INFO  (coreLoadExecutor-5042-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2005945 INFO  (coreLoadExecutor-5042-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-002/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-002/./collection1/data/]
   [junit4]   2> 2005946 INFO  (coreLoadExecutor-5042-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 2005946 INFO  (coreLoadExecutor-5042-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-002/./collection1/data
   [junit4]   2> 2005946 INFO  (coreLoadExecutor-5042-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-002/./collection1/data/index/
   [junit4]   2> 2005946 WARN  (coreLoadExecutor-5042-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-002/./collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2005946 INFO  (coreLoadExecutor-5042-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-002/./collection1/data/index
   [junit4]   2> 2005946 INFO  (coreLoadExecutor-5042-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=22, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8917305865821082]
   [junit4]   2> 2005947 INFO  (coreLoadExecutor-5042-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@23b89cf4 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2d48697e),segFN=segments_1,generation=1}
   [junit4]   2> 2005947 INFO  (coreLoadExecutor-5042-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2005949 INFO  (coreLoadExecutor-5042-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2005952 INFO  (coreLoadExecutor-5042-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2005954 INFO  (coreLoadExecutor-5042-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2005955 INFO  (coreLoadExecutor-5042-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2005956 INFO  (coreLoadExecutor-5042-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2005957 INFO  (coreLoadExecutor-5042-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 2005957 INFO  (coreLoadExecutor-5042-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /schema,/admin/ping,/admin/system,/update,/update/json/docs,/config,/admin/mbeans,lazy,/admin/properties,/admin/plugins,/admin/file,/update/csv,/admin/threads,/admin/luke,/admin/segments,defaults,/get,/admin/logging,standard,/update/json,/replication
   [junit4]   2> 2005959 INFO  (coreLoadExecutor-5042-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2005959 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-002/./collection1/data/
   [junit4]   2> 2005959 INFO  (coreLoadExecutor-5042-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2005959 INFO  (coreLoadExecutor-5042-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2005960 INFO  (coreLoadExecutor-5042-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=2, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 2005960 INFO  (coreLoadExecutor-5042-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@23b89cf4 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2d48697e),segFN=segments_1,generation=1}
   [junit4]   2> 2005960 INFO  (coreLoadExecutor-5042-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2005960 INFO  (coreLoadExecutor-5042-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@3cf93cf2[collection1] main
   [junit4]   2> 2005960 INFO  (coreLoadExecutor-5042-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-002/collection1/conf
   [junit4]   2> 2005960 INFO  (coreLoadExecutor-5042-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-002/collection1/conf}
   [junit4]   2> 2005960 INFO  (coreLoadExecutor-5042-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_D9B0A5512A1C7560-001/solr-instance-002/collection1/conf
   [junit4]   2> 2005961 INFO  (coreLoadExecutor-5042-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2005963 INFO  (coreLoadExecutor-5042-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms
   [junit4]   2> 2005963 INFO  (coreLoadExecutor-5042-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2005963 INFO  (searcherExecutor-5043-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3cf93cf2[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2005970 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2006293 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/select params={wt=javabin&sort=id+desc&q=*:*&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 2006296 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[0]} 0 0
   [junit4]   2> 2006298 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1]} 0 0
   [junit4]   2> 2006300 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2]} 0 0
   [junit4]   2> 2006302 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[3]} 0 0
   [junit4]   2> 2006303 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[4]} 0 0
   [junit4]   2> 2006305 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[5]} 0 0
   [junit4]   2> 2006307 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[6]} 0 0
   [junit4]   2> 2006309 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[7]} 0 0
   [junit4]   2> 2006311 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[8]} 0 0
   [junit4]   2> 2006313 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[9]} 0 0
   [junit4]   2> 2006315 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[10]} 0 0
   [junit4]   2> 2006317 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[11]} 0 0
   [junit4]   2> 2006318 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[12]} 0 0
   [junit4]   2> 2006320 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[13]} 0 0
   [junit4]   2> 2006322 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[14]} 0 0
   [junit4]   2> 2006324 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[15]} 0 0
   [junit4]   2> 2006326 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[16]} 0 0
   [junit4]   2> 2006328 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[17]} 0 0
   [junit4]   2> 2006330 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[18]} 0 0
   [junit4]   2> 2006333 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[19]} 0 0
   [junit4]   2> 2006335 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[20]} 0 0
   [junit4]   2> 2006337 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[21]} 0 0
   [junit4]   2> 2006339 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[22]} 0 0
   [junit4]   2> 2006341 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[23]} 0 0
   [junit4]   2> 2006343 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[24]} 0 0
   [junit4]   2> 2006350 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[25]} 0 0
   [junit4]   2> 2006357 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[26]} 0 0
   [junit4]   2> 2006361 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[27]} 0 0
   [junit4]   2> 2006363 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[28]} 0 0
   [junit4]   2> 2006366 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[29]} 0 0
   [junit4]   2> 2006370 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[30]} 0 0
   [junit4]   2> 2006372 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[31]} 0 0
   [junit4]   2> 2006377 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[32]} 0 0
   [junit4]   2> 2006379 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[33]} 0 0
   [junit4]   2> 2006381 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[34]} 0 0
   [junit4]   2> 2006383 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[35]} 0 0
   [junit4]   2> 2006385 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[36]} 0 0
   [junit4]   2> 2006389 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[37]} 0 0
   [junit4]   2> 2006394 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[38]} 0 0
   [junit4]   2> 2006396 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[39]} 0 0
   [junit4]   2> 2006398 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[40]} 0 0
   [junit4]   2> 2006400 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[41]} 0 0
   [junit4]   2> 2006402 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[42]} 0 0
   [junit4]   2> 2006406 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[43]} 0 0
   [junit4]   2> 2006408 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[44]} 0 0
   [junit4]   2> 2006411 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[45]} 0 0
   [junit4]   2> 2006414 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[46]} 0 0
   [junit4]   2> 2006417 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[47]} 0 0
   [junit4]   2> 2006419 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[48]} 0 0
   [junit4]   2> 2006421 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[49]} 0 0
   [junit4]   2> 2006423 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[50]} 0 0
   [junit4]   2> 2006425 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[51]} 0 0
   [junit4]   2> 2006427 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[52]} 0 0
   [junit4]   2> 2006438 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[53]} 0 0
   [junit4]   2> 2006440 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[54]} 0 0
   [junit4]   2> 2006443 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[55]} 0 0
   [junit4]   2> 2006446 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[56]} 0 0
   [junit4]   2> 2006448 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[57]} 0 0
   [junit4]   2> 2006450 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[58]} 0 0
   [junit4]   2> 2006454 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[59]} 0 0
   [junit4]   2> 2006456 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[60]} 0 0
   [junit4]   2> 2006458 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[61]} 0 0
   [junit4]   2> 2006460 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[62]} 0 0
   [junit4]   2> 2006462 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[63]} 0 0
   [junit4]   2> 2006464 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[64]} 0 0
   [junit4]   2> 2006467 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[65]} 0 0
   [junit4]   2> 2006471 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[66]} 0 0
   [junit4]   2> 2006473 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[67]} 0 0
   [junit4]   2> 2006475 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[68]} 0 0
   [junit4]   2> 2006478 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[69]} 0 0
   [junit4]   2> 2006480 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[70]} 0 0
   [junit4]   2> 2006482 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[71]} 0 0
   [junit4]   2> 2006484 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[72]} 0 0
   [junit4]   2> 2006491 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[73]} 0 0
   [junit4]   2> 2006494 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[74]} 0 0
   [junit4]   2> 2006496 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[75]} 0 0
   [junit4]   2> 2006500 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[76]} 0 0
   [junit4]   2> 2006502 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[77]} 0 0
   [junit4]   2> 2006510 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[78]} 0 0
   [junit4]   2> 2006513 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[79]} 0 0
   [junit4]   2> 2006515 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[80]} 0 0
   [junit4]   2> 2006518 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[81]} 0 0
   [junit4]   2> 2006520 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[82]} 0 0
   [junit4]   2> 2006522 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[83]} 0 0
   [junit4]   2> 2006525 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[84]} 0 0
   [junit4]   2> 2006528 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[85]} 0 0
   [junit4]   2> 2006531 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[86]} 0 0
   [junit4]   2> 2006533 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[87]} 0 0
   [junit4]   2> 2006535 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[88]} 0 0
   [junit4]   2> 2006537 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[89]} 0 0
   [junit4]   2> 2006538 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[90]} 0 0
   [junit4]   2> 2006540 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[91]} 0 0
   [junit4]   2> 2006543 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[92]} 0 0
   [junit4]   2> 2006545 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[93]} 0 0
   [junit4]   2> 2006547 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[94]} 0 0
   [junit4]   2> 2006550 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[95]} 0 0
   [junit4]   2> 2006552 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[96]} 0 0
   [junit4]   2> 2006554 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[97]} 0 0
   [junit4]   2> 2006557 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[98]} 0 0
   [junit4]   2> 2006558 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[99]} 0 0
   [junit4]   2> 2006560 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[100]} 0 0
   [junit4]   2> 2006563 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[101]} 0 0
   [junit4]   2> 2006565 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[102]} 0 0
   [junit4]   2> 2006568 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[103]} 0 0
   [junit4]   2> 2006570 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[104]} 0 0
   [junit4]   2> 2006572 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[105]} 0 0
   [junit4]   2> 2006574 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[106]} 0 0
   [junit4]   2> 2006576 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[107]} 0 0
   [junit4]   2> 2006578 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[108]} 0 0
   [junit4]   2> 2006580 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[109]} 0 0
   [junit4]   2> 2006581 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[110]} 0 0
   [junit4]   2> 2006583 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[111]} 0 0
   [junit4]   2> 2006585 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[112]} 0 0
   [junit4]   2> 2006588 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[113]} 0 0
   [junit4]   2> 2006590 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[114]} 0 0
   [junit4]   2> 2006592 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[115]} 0 0
   [junit4]   2> 2006594 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[116]} 0 0
   [junit4]   2> 2006596 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[117]} 0 0
   [junit4]   2> 2006598 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[118]} 0 0
   [junit4]   2> 2006600 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[119]} 0 0
   [junit4]   2> 2006602 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[120]} 0 0
   [junit4]   2> 2006604 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[121]} 0 0
   [junit4]   2> 2006606 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[122]} 0 0
   [junit4]   2> 2006608 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[123]} 0 0
   [junit4]   2> 2006610 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[124]} 0 0
   [junit4]   2> 2006612 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[125]} 0 0
   [junit4]   2> 2006613 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[126]} 0 0
   [junit4]   2> 2006616 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[127]} 0 0
   [junit4]   2> 2006618 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[128]} 0 0
   [junit4]   2> 2006620 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[129]} 0 0
   [junit4]   2> 2006622 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[130]} 0 0
   [junit4]   2> 2006624 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[131]} 0 0
   [junit4]   2> 2006626 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[132]} 0 0
   [junit4]   2> 2006628 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[133]} 0 0
   [junit4]   2> 2006630 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[134]} 0 0
   [junit4]   2> 2006632 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[135]} 0 0
   [junit4]   2> 2006634 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[136]} 0 0
   [junit4]   2> 2006636 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[137]} 0 0
   [junit4]   2> 2006638 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[138]} 0 0
   [junit4]   2> 2006643 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[139]} 0 0
   [junit4]   2> 2006645 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[140]} 0 0
   [junit4]   2> 2006647 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[141]} 0 0
   [junit4]   2> 2006649 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[142]} 0 0
   [junit4]   2> 2006651 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[143]} 0 0
   [junit4]   2> 2006653 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[144]} 0 0
   [junit4]   2> 2006654 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[145]} 0 0
   [junit4]   2> 2006656 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[146]} 0 0
   [junit4]   2> 2006658 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[147]} 0 0
   [junit4]   2> 2006660 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[148]} 0 0
   [junit4]   2> 2006662 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[149]} 0 0
   [junit4]   2> 2006665 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[150]} 0 0
   [junit4]   2> 2006666 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[151]} 0 0
   [junit4]   2> 2006670 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[152]} 0 0
   [junit4]   2> 2006674 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[153]} 0 0
   [junit4]   2> 2006676 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[154]} 0 0
   [junit4]   2> 2006679 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[155]} 0 0
   [junit4]   2> 2006681 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[156]} 0 0
   [junit4]   2> 2006683 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[157]} 0 0
   [junit4]   2> 2006685 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[158]} 0 0
   [junit4]   2> 2006688 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[159]} 0 0
   [junit4]   2> 2006689 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[160]} 0 0
   [junit4]   2> 2006691 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[161]} 0 0
   [junit4]   2> 2006693 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[162]} 0 0
   [junit4]   2> 2006695 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[163]} 0 0
   [junit4]   2> 2006697 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[164]} 0 0
   [junit4]   2> 2006699 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[165]} 0 0
   [junit4]   2> 2006701 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[166]} 0 0
   [junit4]   2> 2006703 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[167]} 0 0
   [junit4]   2> 2006705 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[168]} 0 0
   [junit4]   2> 2006707 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[169]} 0 0
   [junit4]   2> 2006709 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[170]} 0 0
   [junit4]   2> 2006711 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[171]} 0 0
   [junit4]   2> 2006713 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[172]} 0 0
   [junit4]   2> 2006714 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[173]} 0 0
   [junit4]   2> 2006716 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[174]} 0 0
   [junit4]   2> 2006718 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[175]} 0 0
   [junit4]   2> 2006720 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[176]} 0 0
   [junit4]   2> 2006723 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[177]} 0 0
   [junit4]   2> 2006725 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[178]} 0 0
   [junit4]   2> 2006728 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[179]} 0 0
   [junit4]   2> 2006730 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[180]} 0 0
   [junit4]   2> 2006732 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[181]} 0 0
   [junit4]   2> 2006734 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[182]} 0 0
   [junit4]   2> 2006736 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[183]} 0 0
   [junit4]   2> 2006738 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[184]} 0 0
   [junit4]   2> 2006741 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[185]} 0 0
   [junit4]   2> 2006742 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[186]} 0 0
   [junit4]   2> 2006744 INFO  (qtp541138062-16252) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[187]} 0 0
   [junit4]   2> 2006746 INFO  (qtp541138062-16248) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[188]} 0 0
   [junit4]   2> 2006749 INFO  (qtp541138062-16250) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[189]} 0 0
   [junit4]   2> 2006751 INFO  (qtp541138062-16249) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[190]} 0 0
   [junit4]   2> 2006753 INFO  (qtp541138062-16251) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=jav

[...truncated too long message...]

ask org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1@6713c309 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@4ff619e3[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 1355]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:215)
   [junit4]   2> 	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:261)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> 6056001 ERROR (zkCallback-1423-thread-30-processing-n:127.0.0.1:39723_m_mn%2Fw-EventThread) [n:127.0.0.1:39723_m_mn%2Fw c:awholynewstresscollection_collection3_2 s:shard3 r:core_node8 x:awholynewstresscollection_collection3_2_shard3_replica3] o.a.z.ClientCnxn Error while calling watcher 
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1@2adfa25b rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@4ff619e3[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 1355]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:215)
   [junit4]   2> 	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:261)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> 6056002 ERROR (zkCallback-1423-thread-30-processing-n:127.0.0.1:39723_m_mn%2Fw-EventThread) [n:127.0.0.1:39723_m_mn%2Fw c:awholynewstresscollection_collection3_2 s:shard3 r:core_node8 x:awholynewstresscollection_collection3_2_shard3_replica3] o.a.z.ClientCnxn Error while calling watcher 
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1@75062b97 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@4ff619e3[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 1355]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:215)
   [junit4]   2> 	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:261)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> 6056002 ERROR (zkCallback-1423-thread-30-processing-n:127.0.0.1:39723_m_mn%2Fw-EventThread) [n:127.0.0.1:39723_m_mn%2Fw c:awholynewstresscollection_collection3_2 s:shard3 r:core_node8 x:awholynewstresscollection_collection3_2_shard3_replica3] o.a.z.ClientCnxn Error while calling watcher 
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1@4de0ce rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@4ff619e3[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 1355]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:215)
   [junit4]   2> 	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:261)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> NOTE: test params are: codec=Lucene54, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=mt_MT, timezone=Pacific/Truk
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=149428488,total=447741952
   [junit4]   2> NOTE: All tests run in this JVM: [RAMDirectoryFactoryTest, ForceLeaderTest, TestFoldingMultitermQuery, BufferStoreTest, TestSchemaVersionResource, TestExclusionRuleCollectionAccess, TestBinaryField, TestInitQParser, TestStressUserVersions, TestSolrCLIRunExample, TestLRUCache, SSLMigrationTest, FileBasedSpellCheckerTest, TestRandomFaceting, CurrencyFieldOpenExchangeTest, TestClassicSimilarityFactory, HdfsChaosMonkeyNothingIsSafeTest, ResourceLoaderTest, TestRestoreCore, BasicAuthIntegrationTest, TestAtomicUpdateErrorCases, CopyFieldTest, RulesTest, SignatureUpdateProcessorFactoryTest, TestExactSharedStatsCache, CacheHeaderTest, JavabinLoaderTest, BasicDistributedZkTest, SpatialRPTFieldTypeTest, HLLSerializationTest, TestRTGBase, PathHierarchyTokenizerFactoryTest, TestBadConfig, ZkNodePropsTest, TestValueSourceCache, PolyFieldTest, EchoParamsTest, SuggestComponentContextFilterQueryTest, DeleteInactiveReplicaTest, TestMiniSolrCloudClusterSSL, VersionInfoTest, DeleteShardTest, SOLR749Test, DefaultValueUpdateProcessorTest, BJQParserTest, SharedFSAutoReplicaFailoverTest, AliasIntegrationTest, PingRequestHandlerTest, ZkStateReaderTest, TestLMDirichletSimilarityFactory, SynonymTokenizerTest, LeaderElectionIntegrationTest, TestShardHandlerFactory, PrimUtilsTest, CloudExitableDirectoryReaderTest, TestComponentsName, HdfsNNFailoverTest, TestSolr4Spatial, TestDynamicFieldResource, DateFieldTest, QueryElevationComponentTest, MultiThreadedOCPTest, JsonLoaderTest, ExternalFileFieldSortTest, PreAnalyzedFieldTest, TestAnalyzedSuggestions, TestCryptoKeys, LeaderInitiatedRecoveryOnCommitTest, TestEmbeddedSolrServerConstructors, TestCopyFieldCollectionResource, TestFaceting, ReplicationFactorTest, RankQueryTest, DateRangeFieldTest, AlternateDirectoryTest, PeerSyncTest, TestStressLucene, CloneFieldUpdateProcessorFactoryTest, OverseerTaskQueueTest, TestArbitraryIndexDir, HdfsSyncSliceTest, HdfsDirectoryFactoryTest, DistribJoinFromCollectionTest, TestRemoteStreaming, TestDocSet, JSONWriterTest, ReturnFieldsTest, TestHighlightDedupGrouping, TestManagedSchemaFieldResource, UUIDFieldTest, TestSolrQueryParserResource, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest]
   [junit4] Completed [555/555 (3!)] on J0 in 4205.73s, 1 test, 1 error <<< FAILURES!

[...truncated 1 lines...]
   [junit4] JVM J0: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J0-20151222_231316_845.sysout
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid11986.hprof ...
   [junit4] Heap dump file created [610126745 bytes in 10.905 secs]
   [junit4] <<< JVM J0: EOF ----

[...truncated 12 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1456: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1010: There were test failures: 555 suites (7 ignored), 2208 tests, 2 errors, 1 failure, 33 ignored (18 assumptions) [seed: D9B0A5512A1C7560]

Total time: 240 minutes 55 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1053 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1053/

All tests passed

Build Log:
[...truncated 12083 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151221_234910_151.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid8961.hprof ...
   [junit4] Heap dump file created [631525320 bytes in 6.549 secs]
   [junit4] <<< JVM J1: EOF ----

   [junit4] JVM J1: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151221_234910_151.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] WARN: Unhandled exception in event serialization. -> java.lang.OutOfMemoryError: Java heap space
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] ERROR: JVM J1 ended with an exception, command line: /x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/jre/bin/java -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps -XX:MaxPermSize=192m -ea -esa -Dtests.prefix=tests -Dtests.seed=A1A17E0C245E1FBB -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.luceneMatchVersion=5.5.0 -Dtests.cleanthreads=perClass -Djava.util.logging.config.file=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/tools/junit4/logging.properties -Dtests.nightly=true -Dtests.weekly=false -Dtests.monster=false -Dtests.slow=true -Dtests.asserts=true -Dtests.multiplier=2 -DtempDir=./temp -Djava.io.tmpdir=./temp -Djunit4.tempDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp -Dcommon.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene -Dclover.db.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/clover/db -Djava.security.policy=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/tools/junit4/solr-tests.policy -Dtests.LUCENE_VERSION=5.5.0 -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Djdk.map.althashing.threshold=0 -Djunit4.childvm.cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1 -Djunit4.childvm.id=1 -Djunit4.childvm.count=3 -Dtests.leaveTemporary=false -Dtests.filterstacks=true -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -Dfile.encoding=UTF-8 -classpath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-test-framework/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/test-framework/lib/junit4-ant-2.3.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/test-framework/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/codecs/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-solrj/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/common/lucene-analyzers-common-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/codecs/lucene-codecs-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/backward-codecs/lucene-backward-codecs-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/highlighter/lucene-highlighter-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/memory/lucene-memory-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/misc/lucene-misc-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/spatial/lucene-spatial-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/expressions/lucene-expressions-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/suggest/lucene-suggest-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/grouping/lucene-grouping-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/queries/lucene-queries-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/queryparser/lucene-queryparser-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/join/lucene-join-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/sandbox/lucene-sandbox-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/antlr4-runtime-4.5.1-1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/asm-5.0.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/asm-commons-5.0.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-cli-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-codec-1.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-configuration-1.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-exec-1.3.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-fileupload-1.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/commons-lang-2.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/concurrentlinkedhashmap-lru-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/dom4j-1.6.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/guava-14.0.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/hadoop-annotations-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/hadoop-auth-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/hadoop-common-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/hadoop-hdfs-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/hppc-0.7.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/htrace-core-3.0.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/jackson-core-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/jackson-dataformat-smile-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/joda-time-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/log4j-1.2.17.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/org.restlet-2.3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/org.restlet.ext.servlet-2.3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/protobuf-java-2.5.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/slf4j-log4j12-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/spatial4j-0.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/lib/t-digest-3.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/commons-io-2.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/httpclient-4.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/httpcore-4.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/httpmime-4.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/noggit-0.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/slf4j-api-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/stax2-api-3.1.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/woodstox-core-asl-4.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/solrj/lib/zookeeper-3.4.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/javax.servlet-api-3.1.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-continuation-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-deploy-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-http-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-io-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-jmx-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-rewrite-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-security-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-server-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-servlet-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-servlets-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-util-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-webapp-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/server/lib/jetty-xml-9.2.13.v20150730.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/example/example-DIH/solr/db/lib/derby-10.9.1.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/test-framework/lib/junit-4.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/test-framework/lib/randomizedtesting-runner-2.3.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/antlr-2.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/antlr4-runtime-4.5.1-1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-core-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-core-api-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-core-avl-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-core-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-i18n-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptor-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-admin-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.2.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151221_234910_151.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151221_234910_151.suites -stdin
   [junit4] ERROR: JVM J1 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1456: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1010: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 296 minutes 44 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1052 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1052/

1 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=640, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=640, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:57652/_s/v: collection already exists: awholynewstresscollection_collection2_6
	at __randomizedtesting.SeedInfo.seed([793D49DE9DD5BA4]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:881)




Build Log:
[...truncated 10320 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/init-core-data-001
   [junit4]   2> 21343 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[793D49DE9DD5BA4]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 21343 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[793D49DE9DD5BA4]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_s/v
   [junit4]   2> 21360 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 21369 INFO  (Thread-69) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 21378 INFO  (Thread-69) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 21468 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.ZkTestServer start zk server on port:42872
   [junit4]   2> 21489 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 21536 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 21574 INFO  (zkCallback-12-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@71a477df name:ZooKeeperConnection Watcher:127.0.0.1:42872 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 21574 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 21575 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 21577 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 21612 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 21612 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 21614 INFO  (zkCallback-13-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@22f30399 name:ZooKeeperConnection Watcher:127.0.0.1:42872/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 21614 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 21615 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 21615 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 21621 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 21628 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 21631 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 21635 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 21635 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 21647 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 21647 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 21650 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 21651 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 21657 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 21658 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 21661 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 21661 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 21669 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 21669 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 21672 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 21672 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 21676 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 21676 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 21679 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 21679 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 21685 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 21686 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 21694 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 21694 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 21702 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 21706 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 21708 INFO  (zkCallback-14-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@378fc5db name:ZooKeeperConnection Watcher:127.0.0.1:42872 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 21708 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 21709 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 21709 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 21722 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 21732 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 21733 INFO  (zkCallback-15-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3e4e92e3 name:ZooKeeperConnection Watcher:127.0.0.1:42872/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 21733 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 21734 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 21734 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 21735 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 21748 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 21749 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 21752 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 21752 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 21756 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 21756 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 21759 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 21760 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 21769 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 21770 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 21773 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 21773 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 21780 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 21780 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 21783 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 21784 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 21787 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 21788 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 21792 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 21793 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 21796 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 21796 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 22138 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/control-001/cores/collection1
   [junit4]   2> 22140 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 22147 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1fb2a275{/_s/v,null,AVAILABLE}
   [junit4]   2> 22158 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.e.j.s.ServerConnector Started ServerConnector@7aab57e8{HTTP/1.1}{127.0.0.1:57652}
   [junit4]   2> 22163 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.e.j.s.Server Started @25056ms
   [junit4]   2> 22163 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=57652, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/control-001/cores, hostContext=/_s/v, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/tempDir-001/control/data}
   [junit4]   2> 22163 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 22164 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/control-001'
   [junit4]   2> 22164 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 22164 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 22164 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 22167 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 22169 INFO  (zkCallback-16-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@76f27248 name:ZooKeeperConnection Watcher:127.0.0.1:42872/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 22169 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 22170 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 22174 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 22174 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/control-001/solr.xml
   [junit4]   2> 22194 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/control-001/cores
   [junit4]   2> 22195 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.CoreContainer New CoreContainer 1961379983
   [junit4]   2> 22195 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/control-001]
   [junit4]   2> 22195 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/control-001/lib
   [junit4]   2> 22195 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 22197 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 22197 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 22197 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 22197 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42872/solr
   [junit4]   2> 22211 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 22211 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 22212 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 22214 INFO  (zkCallback-18-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@27a21c85 name:ZooKeeperConnection Watcher:127.0.0.1:42872 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 22214 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 22215 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 22223 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 22225 INFO  (zkCallback-19-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3c3436ed name:ZooKeeperConnection Watcher:127.0.0.1:42872/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 22225 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 22234 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 22243 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 22249 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 22253 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 22261 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 22273 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 22276 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 22279 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 22281 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 22283 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57652__s%2Fv
   [junit4]   2> 22286 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:57652__s%2Fv
   [junit4]   2> 22292 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 22295 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 22297 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 22299 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95070198549184518-127.0.0.1:57652__s%2Fv-n_0000000000
   [junit4]   2> 22303 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:57652__s%2Fv
   [junit4]   2> 22303 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 22306 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.Overseer Overseer (id=95070198549184518-127.0.0.1:57652__s%2Fv-n_0000000000) starting
   [junit4]   2> 22313 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 22343 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 22347 INFO  (OverseerCollectionConfigSetProcessor-95070198549184518-127.0.0.1:57652__s%2Fv-n_0000000000) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 22347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 22359 INFO  (OverseerStateUpdate-95070198549184518-127.0.0.1:57652__s%2Fv-n_0000000000) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 22459 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 22459 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 22461 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/control-001/cores
   [junit4]   2> 22463 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {shard=, loadOnStartup=true, config=solrconfig.xml, dataDir=data/, configSetProperties=configsetprops.json, collection=control_collection, name=collection1, schema=schema.xml, coreNodeName=, transient=false}
   [junit4]   2> 22463 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/control-001/cores/collection1
   [junit4]   2> 22463 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 22465 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 22465 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 22466 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 22467 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 22475 INFO  (OverseerStateUpdate-95070198549184518-127.0.0.1:57652__s%2Fv-n_0000000000) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:57652/_s/v",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:57652__s%2Fv",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null} current state version: 0
   [junit4]   2> 22483 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 22486 INFO  (OverseerStateUpdate-95070198549184518-127.0.0.1:57652__s%2Fv-n_0000000000) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:57652/_s/v",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:57652__s%2Fv",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null}
   [junit4]   2> 22488 INFO  (OverseerStateUpdate-95070198549184518-127.0.0.1:57652__s%2Fv-n_0000000000) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 22499 INFO  (OverseerStateUpdate-95070198549184518-127.0.0.1:57652__s%2Fv-n_0000000000) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 22504 INFO  (zkCallback-19-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 23484 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 23484 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 23485 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 23485 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 23486 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 23487 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/control-001/cores/collection1'
   [junit4]   2> 23487 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 23487 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 23491 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 23501 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 23517 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 23553 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 23576 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 23860 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 23871 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 23873 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 23895 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 23899 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 23903 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 23904 WARN  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 23904 WARN  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 23905 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 23906 WARN  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 23906 WARN  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 23911 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 23911 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 23911 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 23912 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/control-001/cores/collection1/data/]
   [junit4]   2> 23912 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@66958dc1
   [junit4]   2> 23916 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/control-001/cores/collection1/data
   [junit4]   2> 23916 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/control-001/cores/collection1/data/index/
   [junit4]   2> 23916 WARN  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 23916 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/control-001/cores/collection1/data/index
   [junit4]   2> 23917 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=29, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=2.0732421875, floorSegmentMB=0.7265625, forceMergeDeletesPctAllowed=23.696417039269093, segmentsPerTier=24.0, maxCFSSegmentSizeMB=1.849609375, noCFSRatio=0.7162619745653881
   [junit4]   2> 23918 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@5e4e1db5 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5a808671; maxCacheMB=0.4892578125 maxMergeSizeMB=0.5263671875)),segFN=segments_1,generation=1}
   [junit4]   2> 23918 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 23927 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/control-001/cores/collection1/data/
   [junit4]   2> 23927 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 23937 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 23946 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 23947 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 23947 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 23947 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 23947 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 23947 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 23948 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 23948 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 23948 WARN  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 23951 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 23953 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 23954 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 23956 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 23969 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/properties,/admin/threads,/admin/file,standard,/admin/mbeans,/get,/schema,/admin/luke,/replication,/update/csv,/update/json/docs,/dump,/config,/admin/system,/update,/admin/segments,/admin/ping,/update/json,/admin/logging,/admin/plugins
   [junit4]   2> 23970 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 23971 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 23971 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 23972 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 23972 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 23973 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=43, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=90.24609375, floorSegmentMB=1.7919921875, forceMergeDeletesPctAllowed=20.201589104336385, segmentsPerTier=46.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1689087673718867
   [junit4]   2> 23973 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@5e4e1db5 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5a808671; maxCacheMB=0.4892578125 maxMergeSizeMB=0.5263671875)),segFN=segments_1,generation=1}
   [junit4]   2> 23974 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 23974 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@451573c3[collection1] main
   [junit4]   2> 23974 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 23976 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 23976 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 23979 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 23979 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 23979 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 23980 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 23980 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 23980 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 23980 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 23982 INFO  (searcherExecutor-130-thread-1-processing-r:core_node1 n:127.0.0.1:57652__s%2Fv x:collection1 c:control_collection s:shard1) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@451573c3[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 23983 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 23983 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 23983 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 23983 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1521123179455578112
   [junit4]   2> 23984 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1521123179455578112
   [junit4]   2> 23986 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 23987 INFO  (coreLoadExecutor-129-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 23988 INFO  (coreZkRegister-123-thread-1-processing-r:core_node1 n:127.0.0.1:57652__s%2Fv x:collection1 c:control_collection s:shard1) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:57652/_s/v collection:control_collection shard:shard1
   [junit4]   2> 23996 INFO  (coreZkRegister-123-thread-1-processing-r:core_node1 n:127.0.0.1:57652__s%2Fv x:collection1 c:control_collection s:shard1) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 24010 INFO  (coreZkRegister-123-thread-1-processing-r:core_node1 n:127.0.0.1:57652__s%2Fv x:collection1 c:control_collection s:shard1) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95070198549184518-core_node1-n_0000000000
   [junit4]   2> 24011 INFO  (coreZkRegister-123-thread-1-processing-r:core_node1 n:127.0.0.1:57652__s%2Fv x:collection1 c:control_collection s:shard1) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=180000
   [junit4]   2> 24015 INFO  (coreZkRegister-123-thread-1-processing-r:core_node1 n:127.0.0.1:57652__s%2Fv x:collection1 c:control_collection s:shard1) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 24015 INFO  (coreZkRegister-123-thread-1-processing-r:core_node1 n:127.0.0.1:57652__s%2Fv x:collection1 c:control_collection s:shard1) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 24016 INFO  (OverseerStateUpdate-95070198549184518-127.0.0.1:57652__s%2Fv-n_0000000000) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 24017 INFO  (coreZkRegister-123-thread-1-processing-r:core_node1 n:127.0.0.1:57652__s%2Fv x:collection1 c:control_collection s:shard1) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:57652/_s/v/collection1/
   [junit4]   2> 24017 INFO  (coreZkRegister-123-thread-1-processing-r:core_node1 n:127.0.0.1:57652__s%2Fv x:collection1 c:control_collection s:shard1) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 24017 INFO  (coreZkRegister-123-thread-1-processing-r:core_node1 n:127.0.0.1:57652__s%2Fv x:collection1 c:control_collection s:shard1) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:57652/_s/v/collection1/ has no replicas
   [junit4]   2> 24027 INFO  (zkCallback-19-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 24028 INFO  (coreZkRegister-123-thread-1-processing-r:core_node1 n:127.0.0.1:57652__s%2Fv x:collection1 c:control_collection s:shard1) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 24037 INFO  (coreZkRegister-123-thread-1-processing-r:core_node1 n:127.0.0.1:57652__s%2Fv x:collection1 c:control_collection s:shard1) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 24054 INFO  (coreZkRegister-123-thread-1-processing-r:core_node1 n:127.0.0.1:57652__s%2Fv x:collection1 c:control_collection s:shard1) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:57652/_s/v/collection1/ shard1
   [junit4]   2> 24056 INFO  (OverseerStateUpdate-95070198549184518-127.0.0.1:57652__s%2Fv-n_0000000000) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:57652/_s/v",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 24161 INFO  (zkCallback-19-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 24207 INFO  (coreZkRegister-123-thread-1-processing-r:core_node1 n:127.0.0.1:57652__s%2Fv x:collection1 c:control_collection s:shard1) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:57652/_s/v/collection1/ and leader is http://127.0.0.1:57652/_s/v/collection1/
   [junit4]   2> 24207 INFO  (coreZkRegister-123-thread-1-processing-r:core_node1 n:127.0.0.1:57652__s%2Fv x:collection1 c:control_collection s:shard1) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57652/_s/v
   [junit4]   2> 24207 INFO  (coreZkRegister-123-thread-1-processing-r:core_node1 n:127.0.0.1:57652__s%2Fv x:collection1 c:control_collection s:shard1) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 24207 INFO  (coreZkRegister-123-thread-1-processing-r:core_node1 n:127.0.0.1:57652__s%2Fv x:collection1 c:control_collection s:shard1) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 24207 INFO  (coreZkRegister-123-thread-1-processing-r:core_node1 n:127.0.0.1:57652__s%2Fv x:collection1 c:control_collection s:shard1) [n:127.0.0.1:57652__s%2Fv c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 24211 INFO  (OverseerStateUpdate-95070198549184518-127.0.0.1:57652__s%2Fv-n_0000000000) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:57652/_s/v",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:57652__s%2Fv",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1"} current state version: 3
   [junit4]   2> 24213 INFO  (OverseerStateUpdate-95070198549184518-127.0.0.1:57652__s%2Fv-n_0000000000) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:57652/_s/v",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:57652__s%2Fv",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1"}
   [junit4]   2> 24318 INFO  (zkCallback-19-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 24482 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 24483 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 24484 INFO  (zkCallback-21-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@706a420d name:ZooKeeperConnection Watcher:127.0.0.1:42872/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 24485 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 24485 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 24485 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 24492 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 24861 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/shard-1-001/cores/collection1
   [junit4]   2> 24862 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/shard-1-001
   [junit4]   2> 24863 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 24866 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@10e728dd{/_s/v,null,AVAILABLE}
   [junit4]   2> 24867 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.e.j.s.ServerConnector Started ServerConnector@43d6b969{HTTP/1.1}{127.0.0.1:41120}
   [junit4]   2> 24867 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.e.j.s.Server Started @27760ms
   [junit4]   2> 24867 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/shard-1-001/cores, hostContext=/_s/v, hostPort=41120}
   [junit4]   2> 24867 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 24868 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/shard-1-001'
   [junit4]   2> 24868 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 24868 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 24868 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 24870 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 24873 INFO  (zkCallback-22-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@72d0e932 name:ZooKeeperConnection Watcher:127.0.0.1:42872/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 24873 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 24873 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 24875 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 24875 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/shard-1-001/solr.xml
   [junit4]   2> 24895 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/shard-1-001/cores
   [junit4]   2> 24895 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.CoreContainer New CoreContainer 772861174
   [junit4]   2> 24895 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/shard-1-001]
   [junit4]   2> 24895 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/shard-1-001/lib
   [junit4]   2> 24896 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 24897 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 24898 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 24898 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 24898 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42872/solr
   [junit4]   2> 24898 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 24898 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 24899 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 24901 INFO  (zkCallback-24-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1dccfb13 name:ZooKeeperConnection Watcher:127.0.0.1:42872 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 24901 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 24902 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 24906 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:41120__s%2Fv    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 24906 INFO  (zkCallback-25-thread-1-processing-n:127.0.0.1:41120__s%2Fv) [n:127.0.0.1:41120__s%2Fv    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7b783d77 name:ZooKeeperConnection Watcher:127.0.0.1:42872/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 24907 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:41120__s%2Fv    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 24921 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:41120__s%2Fv    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 25932 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:41120__s%2Fv    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41120__s%2Fv
   [junit4]   2> 25932 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:41120__s%2Fv    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:41120__s%2Fv
   [junit4]   2> 25935 INFO  (zkCallback-21-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 25935 INFO  (zkCallback-19-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 25935 INFO  (zkCallback-25-thread-1-processing-n:127.0.0.1:41120__s%2Fv) [n:127.0.0.1:41120__s%2Fv    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 25938 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:41120__s%2Fv    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 25939 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:41120__s%2Fv    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95070198549184522-127.0.0.1:41120__s%2Fv-n_0000000001
   [junit4]   2> 25942 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:41120__s%2Fv    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/95070198549184518-127.0.0.1:57652__s%2Fv-n_0000000000 to know if I could be the leader
   [junit4]   2> 25972 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:41120__s%2Fv    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 25973 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:41120__s%2Fv    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 25974 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:41120__s%2Fv    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/shard-1-001/cores
   [junit4]   2> 25975 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:41120__s%2Fv    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, collection=collection1, loadOnStartup=true, schema=schema.xml, dataDir=data/, transient=false, coreNodeName=, shard=, config=solrconfig.xml, configSetProperties=configsetprops.json}
   [junit4]   2> 25975 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:41120__s%2Fv    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/shard-1-001/cores/collection1
   [junit4]   2> 25984 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:41120__s%2Fv    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 25988 INFO  (coreLoadExecutor-140-thread-1-processing-n:127.0.0.1:41120__s%2Fv) [n:127.0.0.1:41120__s%2Fv c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 25988 INFO  (coreLoadExecutor-140-thread-1-processing-n:127.0.0.1:41120__s%2Fv) [n:127.0.0.1:41120__s%2Fv c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 25989 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:41120__s%2Fv    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 25989 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[793D49DE9DD5BA4]) [n:127.0.0.1:41120__s%2Fv    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 25992 INFO  (coreLoadExecutor-140-thread-1-processing-n:127.0.0.1:41120__s%2Fv) [n:127.0.0.1:41120__s%2Fv c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 25993 INFO  (OverseerStateUpdate-95070198549184518-127.0.0.1:57652__s%2Fv-n_0000000000) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:41120/_s/v",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:41120__s%2Fv",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":null} current state version: 4
   [junit4]   2> 25993 INFO  (OverseerStateUpdate-95070198549184518-127.0.0.1:57652__s%2Fv-n_0000000000) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:41120/_s/v",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:41120__s%2Fv",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":null}
   [junit4]   2> 25994 INFO  (OverseerStateUpdate-95070198549184518-127.0.0.1:57652__s%2Fv-n_0000000000) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 25994 INFO  (OverseerStateUpdate-95070198549184518-127.0.0.1:57652__s%2Fv-n_0000000000) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 25997 INFO  (zkCallback-21-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 25997 INFO  (zkCallback-19-thread-1-processing-n:127.0.0.1:57652__s%2Fv) [n:127.0.0.1:57652__s%2Fv    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 25999 INFO  (zkCallback-25-thread-1-processing-n:127.0.0.1:41120__s%2Fv) [n:127.0.0.1:41120__s%2Fv    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 26993 INFO  (coreLoadExecutor-140-thread-1-processing-n:127.0.0.1:41120__s%2Fv) [n:127.0.0.1:41120__s%2Fv c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 26993 INFO  (coreLoadExecutor-140-thread-1-processing-n:127.0.0.1:41120__s%2Fv) [n:127.0.0.1:41120__s%2Fv c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 26994 INFO  (coreLoadExecutor-140-thread-1-processing-n:127.0.0.1:41120__s%2Fv) [n:127.0.0.1:41120__s%2Fv c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 26994 INFO  (coreLoadExecutor-140-thread-1-processing-n:127.0.0.1:41120__s%2Fv) [n:127.0.0.1:41120__s%2Fv c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 26996 INFO  (coreLoadExecutor-140-thread-1-processing-n:127.0.0.1:41120__s%2Fv) [n:127.0.0.1:41120__s%2Fv c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 26996 INFO  (coreLoadExecutor-140-thread-1-processing-n:127.0.0.1:41120__s%2Fv) [n:127.0.0.1:41120__s%2Fv c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_793D49DE9DD5BA4-001/shard-1-001/cores/collection1'
   [junit4]   2> 26996 INFO  (coreLoadExecutor-140-thread-1-processing-n:127.0.0.1:41120__s%2Fv) [n:127.0.0.1:41120__s%2Fv c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 26996 INFO  (coreLoadExecutor-140-thread-1-processing-n:127.0.0.1:41120__s%2Fv) [n:127.0.0.1:41120__s%2Fv c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 26999 INFO  (coreLoadExecutor-140-thread-1-processing-n:127.0.0.1:41120__s%2Fv) [n:127.0.0.1:41120__s%2Fv c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 27010 INFO  (coreLoadExecutor-140-thread-1-processing-n:127.0.0.1:41120__s%2Fv) [n:127.0.0.1:41120__s%2Fv c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 27025 INFO  (coreLoadExecutor-140-thread-1-processing-n:127.0.0.1:41120__s%2Fv) [n:127.0.0.1:41120__s%2Fv c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 27064 INFO  (coreLoadExecutor-140-thread-1-processing-n:127.0.0.1:41120__s%2Fv) [n:127.0.0.1:41120__s%2Fv c:collection1   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 27079 INFO  (coreLoadExecutor-140-thread-1-processing-n:127.0.0.1:41120__s%2Fv) [n:127.0.0.1:41120__s%2Fv c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 27381 INFO  (coreLoadExecutor-140-thread-1-processing-n:127.0.0.1:41120__s%2Fv) [n:127.0.0.1:41120__s%2Fv c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 27392 INFO  (coreLoadExecutor-140-thread-1-processing-n:127.0.0.1:41120__s%2Fv) [n:127.0.0.1:41120__s%2Fv c:collection1   x:colle

[...truncated too long message...]

s/awholynewstresscollection_collection2_6/leader_elect/shard4/election/95070198549184518-core_node31-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection3_1/leader_elect/shard5/election/95070198549184531-core_node12-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection2_6/leader_elect/shard1/election/95070198549184528-core_node1-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection1_6/leader_elect/shard1/election/95070198549184522-core_node22-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection3_5/leader_elect/shard2/election/95070198549184522-core_node9-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection5_6/leader_elect/shard3/election/95070198549184525-core_node22-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection5_6/leader_elect/shard7/election/95070198549184522-core_node4-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection2_3/leader_elect/shard1/election/95070198549184528-core_node4-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection2_1/leader_elect/shard4/election/95070198549184528-core_node5-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewcollection_1/leader_elect/shard2/election/95070198549184522-core_node7-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection2_5/leader_elect/shard5/election/95070198549184518-core_node18-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection1_5/leader_elect/shard3/election/95070198549184528-core_node5-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection1_4/leader_elect/shard5/election/95070198549184531-core_node13-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewcollection_1/leader_elect/shard4/election/95070198549184518-core_node1-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection5_5/leader_elect/shard3/election/95070198549184528-core_node13-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection3_5/leader_elect/shard3/election/95070198549184528-core_node16-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection3_6/leader_elect/shard8/election/95070198549184518-core_node11-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection1_5/leader_elect/shard1/election/95070198549184522-core_node9-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection0_5/leader_elect/shard4/election/95070198549184522-core_node20-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection4_4/leader_elect/shard5/election/95070198549184518-core_node1-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection2_4/leader_elect/shard3/election/95070198549184525-core_node8-n_0000000001
   [junit4]   2> 	2	/solr/collections/halfdeletedcollection2/leader_elect/shard1/election/95070198549184525-core_node2-n_0000000000
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	67	/solr/collections
   [junit4]   2> 	66	/solr/live_nodes
   [junit4]   2> 	3	/solr/overseer/queue
   [junit4]   2> 	3	/solr/overseer/collection-queue-work
   [junit4]   2> 	2	/solr/overseer/queue-work
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=CollectionsAPIDistributedZkTest -Dtests.method=test -Dtests.seed=793D49DE9DD5BA4 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=sq_AL -Dtests.timezone=Asia/Novokuznetsk -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR    456s J1 | CollectionsAPIDistributedZkTest.test <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=640, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:57652/_s/v: collection already exists: awholynewstresscollection_collection2_6
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([793D49DE9DD5BA4]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:881)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=641, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:57652/_s/v: collection already exists: awholynewstresscollection_collection3_6
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([793D49DE9DD5BA4]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:881)
   [junit4]   2> 477594 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[793D49DE9DD5BA4]-worker) [n:127.0.0.1:53118__s%2Fv c:addReplicaColl s:shard2 r:core_node6 x:addReplicaColl_shard2_replica3] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=DefaultSimilarity, locale=sq_AL, timezone=Asia/Novokuznetsk
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=124562024,total=493355008
   [junit4]   2> NOTE: All tests run in this JVM: [SchemaVersionSpecificBehaviorTest, TestManagedSchemaFieldTypeResource, AnalysisErrorHandlingTest, SolrInfoMBeanTest, HighlighterTest, CoreAdminRequestStatusTest, CollectionsAPIDistributedZkTest]
   [junit4] Completed [66/555 (1!)] on J1 in 457.25s, 1 test, 1 error <<< FAILURES!

[...truncated 1578 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1456: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1010: There were test failures: 555 suites (7 ignored), 2206 tests, 1 error, 33 ignored (18 assumptions) [seed: 793D49DE9DD5BA4]

Total time: 222 minutes 16 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any