You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/03/30 22:30:32 UTC

[JENKINS] Lucene-Solr-4.7-Linux (32bit/ibm-j9-jdk7) - Build # 55 - Failure!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.7-Linux/55/
Java: 32bit/ibm-j9-jdk7 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}

1 tests failed.
REGRESSION:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch

Error Message:
deletealias the collection time out:180s

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: deletealias the collection time out:180s
	at __randomizedtesting.SeedInfo.seed([D19CFC9A78C27D96:507A72820F9D1DAA]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:495)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:199)
	at org.apache.solr.client.solrj.request.CollectionAdminRequest.process(CollectionAdminRequest.java:272)
	at org.apache.solr.client.solrj.request.CollectionAdminRequest.deleteAlias(CollectionAdminRequest.java:404)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testSolrJAPICalls(CollectionsAPIDistributedZkTest.java:304)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:199)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:870)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:94)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
	at java.lang.reflect.Method.invoke(Method.java:619)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:853)




Build Log:
[...truncated 11014 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> 649946 T2866 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 649947 T2866 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /cez/fi
   [junit4]   2> 649952 T2866 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1/./solrtest-CollectionsAPIDistributedZkTest-1396210345881
   [junit4]   2> 649953 T2866 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 649954 T2867 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 650054 T2866 oasc.ZkTestServer.run start zk server on port:55386
   [junit4]   2> 650056 T2866 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 650058 T2873 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a68567ca name:ZooKeeperConnection Watcher:127.0.0.1:55386 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 650058 T2866 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 650059 T2866 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 650062 T2866 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 650063 T2875 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4c11730 name:ZooKeeperConnection Watcher:127.0.0.1:55386/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 650063 T2866 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 650064 T2866 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 650067 T2866 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 650069 T2866 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 650071 T2866 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 650073 T2866 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 650074 T2866 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 650078 T2866 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 650079 T2866 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 650082 T2866 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 650083 T2866 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 650086 T2866 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 650087 T2866 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 650089 T2866 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 650090 T2866 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 650091 T2866 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 650092 T2866 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 650094 T2866 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 650094 T2866 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 650097 T2866 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 650097 T2866 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 650099 T2866 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 650100 T2866 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 650102 T2866 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 650102 T2866 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 650104 T2866 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 650105 T2866 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> Using old style solr.xml
   [junit4]   2> 650109 T2866 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 650110 T2877 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@646a0ed7 name:ZooKeeperConnection Watcher:127.0.0.1:55386 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 650110 T2866 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 650110 T2866 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 650113 T2866 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 650114 T2879 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@95e17814 name:ZooKeeperConnection Watcher:127.0.0.1:55386/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 650114 T2866 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 650115 T2866 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 650115 T2866 oascc.SolrZkClient.makePath makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 650119 T2866 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 650120 T2866 oascc.SolrZkClient.makePath makePath: /configs/conf2/schema.xml
   [junit4]   2> 650125 T2866 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 650126 T2866 oascc.SolrZkClient.makePath makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 650128 T2866 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 650129 T2866 oascc.SolrZkClient.makePath makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 650132 T2866 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 650132 T2866 oascc.SolrZkClient.makePath makePath: /configs/conf2/protwords.txt
   [junit4]   2> 650136 T2866 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 650136 T2866 oascc.SolrZkClient.makePath makePath: /configs/conf2/currency.xml
   [junit4]   2> 650139 T2866 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 650140 T2866 oascc.SolrZkClient.makePath makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 650143 T2866 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 650144 T2866 oascc.SolrZkClient.makePath makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 650146 T2866 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 650147 T2866 oascc.SolrZkClient.makePath makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 650149 T2866 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 650150 T2866 oascc.SolrZkClient.makePath makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 650153 T2866 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 650153 T2866 oascc.SolrZkClient.makePath makePath: /configs/conf2/elevate.xml
   [junit4]   2> 650229 T2866 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 650232 T2866 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:40013
   [junit4]   2> 650233 T2866 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 650234 T2866 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 650234 T2866 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1396210346084
   [junit4]   2> 650235 T2866 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1396210346084/'
   [junit4]   2> 650242 T2866 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1396210346084/solr.xml
   [junit4]   2> 650274 T2866 oasc.CoreContainer.<init> New CoreContainer -1169707040
   [junit4]   2> 650274 T2866 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1396210346084/]
   [junit4]   2> 650276 T2866 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 650276 T2866 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
   [junit4]   2> 650277 T2866 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 650277 T2866 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 650278 T2866 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 650278 T2866 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 650279 T2866 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 650279 T2866 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 650280 T2866 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 650280 T2866 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 650281 T2866 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 650281 T2866 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 650282 T2866 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55386/solr
   [junit4]   2> 650282 T2866 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 650283 T2866 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 650285 T2890 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5dde7885 name:ZooKeeperConnection Watcher:127.0.0.1:55386 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 650285 T2866 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 650287 T2866 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 650288 T2892 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5c70c857 name:ZooKeeperConnection Watcher:127.0.0.1:55386/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 650288 T2866 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 650290 T2866 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 650294 T2866 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 650298 T2866 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 650300 T2866 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40013_cez%2Ffi
   [junit4]   2> 650301 T2866 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:40013_cez%2Ffi
   [junit4]   2> 650303 T2866 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 650305 T2866 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 650309 T2866 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:40013_cez%2Ffi
   [junit4]   2> 650309 T2866 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 650311 T2866 oasc.Overseer.start Overseer (id=91502041227853829-127.0.0.1:40013_cez%2Ffi-n_0000000000) starting
   [junit4]   2> 650314 T2866 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 650317 T2894 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 650318 T2866 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 650318 T2894 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
   [junit4]   2> 650320 T2866 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 650322 T2866 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 650324 T2893 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 650326 T2866 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 650327 T2866 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 650328 T2866 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 650329 T2897 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3b2cece3 name:ZooKeeperConnection Watcher:127.0.0.1:55386/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 650329 T2866 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 650330 T2866 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 650332 T2866 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 650415 T2866 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
   [junit4]   2> 650415 T2866 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 650417 T2866 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:48591
   [junit4]   2> 650418 T2866 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 650418 T2866 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 650419 T2866 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1396210346260
   [junit4]   2> 650419 T2866 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1396210346260/'
   [junit4]   2> 650427 T2866 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1396210346260/solr.xml
   [junit4]   2> 650459 T2866 oasc.CoreContainer.<init> New CoreContainer 430787655
   [junit4]   2> 650460 T2866 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1396210346260/]
   [junit4]   2> 650461 T2866 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 650462 T2866 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
   [junit4]   2> 650462 T2866 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 650463 T2866 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 650463 T2866 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 650464 T2866 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 650464 T2866 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 650465 T2866 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 650465 T2866 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 650466 T2866 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 650466 T2866 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 650467 T2866 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 650467 T2866 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55386/solr
   [junit4]   2> 650468 T2866 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 650469 T2866 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 650471 T2908 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5e48c64f name:ZooKeeperConnection Watcher:127.0.0.1:55386 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 650471 T2866 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 650473 T2866 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 650474 T2910 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6e5a2e83 name:ZooKeeperConnection Watcher:127.0.0.1:55386/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 650474 T2866 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 650478 T2866 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 651480 T2866 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48591_cez%2Ffi
   [junit4]   2> 651482 T2866 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:48591_cez%2Ffi
   [junit4]   2> 651485 T2897 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 651485 T2892 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 651485 T2910 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 651489 T2866 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 651490 T2866 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 651553 T2866 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
   [junit4]   2> 651554 T2866 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 651556 T2866 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:33955
   [junit4]   2> 651556 T2866 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 651557 T2866 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 651557 T2866 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1396210347418
   [junit4]   2> 651558 T2866 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1396210347418/'
   [junit4]   2> 651566 T2866 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1396210347418/solr.xml
   [junit4]   2> 651606 T2866 oasc.CoreContainer.<init> New CoreContainer 387774060
   [junit4]   2> 651606 T2866 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1396210347418/]
   [junit4]   2> 651608 T2866 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 651608 T2866 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
   [junit4]   2> 651609 T2866 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 651609 T2866 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 651610 T2866 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 651610 T2866 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 651611 T2866 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 651611 T2866 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 651611 T2866 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 651612 T2866 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 651613 T2866 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 651613 T2866 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 651614 T2866 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55386/solr
   [junit4]   2> 651614 T2866 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 651615 T2866 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 651617 T2922 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@bd434db2 name:ZooKeeperConnection Watcher:127.0.0.1:55386 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 651617 T2866 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 651619 T2866 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 651621 T2924 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@799d9fc name:ZooKeeperConnection Watcher:127.0.0.1:55386/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 651621 T2866 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 651624 T2866 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 652626 T2866 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33955_cez%2Ffi
   [junit4]   2> 652628 T2866 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:33955_cez%2Ffi
   [junit4]   2> 652630 T2897 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 652630 T2892 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 652630 T2910 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 652631 T2924 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 652635 T2866 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 652636 T2866 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 652707 T2866 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
   [junit4]   2> 652708 T2866 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 652710 T2866 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:37752
   [junit4]   2> 652711 T2866 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 652712 T2866 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 652712 T2866 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1396210348565
   [junit4]   2> 652713 T2866 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1396210348565/'
   [junit4]   2> 652722 T2866 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1396210348565/solr.xml
   [junit4]   2> 652753 T2866 oasc.CoreContainer.<init> New CoreContainer -549825451
   [junit4]   2> 652754 T2866 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1396210348565/]
   [junit4]   2> 652755 T2866 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 652756 T2866 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
   [junit4]   2> 652756 T2866 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 652757 T2866 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 652757 T2866 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 652758 T2866 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 652758 T2866 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 652759 T2866 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 652759 T2866 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 652760 T2866 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 652760 T2866 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 652761 T2866 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 652762 T2866 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55386/solr
   [junit4]   2> 652762 T2866 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 652763 T2866 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 652764 T2936 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5d877411 name:ZooKeeperConnection Watcher:127.0.0.1:55386 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 652764 T2866 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 652766 T2866 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 652767 T2938 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e1c8c7d4 name:ZooKeeperConnection Watcher:127.0.0.1:55386/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 652768 T2866 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 652771 T2866 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 653774 T2866 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37752_cez%2Ffi
   [junit4]   2> 653776 T2866 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:37752_cez%2Ffi
   [junit4]   2> 653779 T2897 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 653779 T2924 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 653779 T2910 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 653780 T2938 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 653780 T2892 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 653785 T2866 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 653786 T2866 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 653850 T2866 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 4
   [junit4]   2> 653851 T2866 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 653853 T2866 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:39330
   [junit4]   2> 653854 T2866 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 653855 T2866 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 653855 T2866 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1396210349715
   [junit4]   2> 653856 T2866 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1396210349715/'
   [junit4]   2> 653863 T2866 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1396210349715/solr.xml
   [junit4]   2> 653895 T2866 oasc.CoreContainer.<init> New CoreContainer 729784436
   [junit4]   2> 653895 T2866 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1396210349715/]
   [junit4]   2> 653897 T2866 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 653897 T2866 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
   [junit4]   2> 653898 T2866 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 653898 T2866 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 653899 T2866 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 653899 T2866 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 653900 T2866 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 653900 T2866 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 653901 T2866 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 653901 T2866 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 653902 T2866 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 653902 T2866 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 653903 T2866 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55386/solr
   [junit4]   2> 653904 T2866 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 653905 T2866 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 653907 T2950 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@da351d5c name:ZooKeeperConnection Watcher:127.0.0.1:55386 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 653907 T2866 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 653909 T2866 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 653911 T2952 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@aea8a23d name:ZooKeeperConnection Watcher:127.0.0.1:55386/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 653911 T2866 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 653914 T2866 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 654917 T2866 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39330_cez%2Ffi
   [junit4]   2> 654919 T2866 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:39330_cez%2Ffi
   [junit4]   2> 654921 T2897 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 654922 T2924 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 654922 T2952 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 654921 T2938 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 654921 T2892 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 654922 T2910 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 654926 T2866 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 654927 T2866 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 654932 T2905 oasha.CollectionsHandler.handleCreateAction Creating Collection : router.field=myOwnField&replicationFactor=2&numShards=2&router.name=compositeId&action=CREATE&collection.configName=conf1&name=solrj_collection&version=2&wt=javabin
   [junit4]   2> 654934 T2892 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 654935 T2894 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
   [junit4]   2> 	  "operation":"createcollection",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"solrj_collection",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "collection.configName":"conf1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "router.field":"myOwnField",
   [junit4]   2> 	  "router.name":"compositeId"}
   [junit4]   2> 654935 T2894 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : createcollection , {
   [junit4]   2> 	  "operation":"createcollection",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"solrj_collection",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "collection.configName":"conf1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "router.field":"myOwnField",
   [junit4]   2> 	  "router.name":"compositeId"}
   [junit4]   2> 654936 T2894 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/solrj_collection 
   [junit4]   2> 654936 T2894 oascc.SolrZkClient.makePath makePath: /collections/solrj_collection
   [junit4]   2> 654939 T2892 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 654940 T2893 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 654940 T2893 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: solrj_collection
   [junit4]   2> 654941 T2893 oasc.Overseer$ClusterStateUpdater.createCollection Create collection solrj_collection with shards [shard1, shard2]
   [junit4]   2> 654942 T2892 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 654943 T2924 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 654943 T2892 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 654944 T2938 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 654943 T2952 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 654943 T2910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 654943 T2897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 655039 T2894 oasc.OverseerCollectionProcessor.createCollection going to create cores replicas shardNames [shard1, shard2] , repFactor : 2
   [junit4]   2> 655039 T2894 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard1_replica1 as part of slice shard1 of collection solrj_collection on 127.0.0.1:37752_cez%2Ffi
   [junit4]   2> 655040 T2894 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard1_replica2 as part of slice shard1 of collection solrj_collection on 127.0.0.1:39330_cez%2Ffi
   [junit4]   2> 655041 T2894 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard2_replica1 as part of slice shard2 of collection solrj_collection on 127.0.0.1:40013_cez%2Ffi
   [junit4]   2> 655042 T2894 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard2_replica2 as part of slice shard2 of collection solrj_collection on 127.0.0.1:33955_cez%2Ffi
   [junit4]   2> 655042 T2945 oasc.ZkController.publish publishing core=solrj_collection_shard1_replica2 state=down collection=solrj_collection
   [junit4]   2> 655043 T2919 oasc.ZkController.publish publishing core=solrj_collection_shard2_replica2 state=down collection=solrj_collection
   [junit4]   2> 655044 T2892 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 655042 T2933 oasc.ZkController.publish publishing core=solrj_collection_shard1_replica1 state=down collection=solrj_collection
   [junit4]   2> 655043 T2886 oasc.ZkController.publish publishing core=solrj_collection_shard2_replica1 state=down collection=solrj_collection
   [junit4]   2> 655045 T2919 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 655046 T2892 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 655046 T2893 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 655044 T2945 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 655047 T2893 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:39330/cez/fi",
   [junit4]   2> 	  "core":"solrj_collection_shard1_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:39330_cez%2Ffi",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"solrj_collection",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 655046 T2886 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 655046 T2933 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 655049 T2893 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:33955/cez/fi",
   [junit4]   2> 	  "core":"solrj_collection_shard2_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:33955_cez%2Ffi",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"solrj_collection",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 655051 T2892 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 655052 T2893 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:37752/cez/fi",
   [junit4]   2> 	  "core":"solrj_collection_shard1_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:37752_cez%2Ffi",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"solrj_collection",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 655054 T2892 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 655055 T2893 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:40013/cez/fi",
   [junit4]   2> 	  "core":"solrj_collection_shard2_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:40013_cez%2Ffi",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"solrj_collection",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 655057 T2892 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 655159 T2897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 655159 T2892 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 655159 T2938 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 655159 T2924 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 655159 T2952 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 655159 T2910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 656046 T2919 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard2_replica2
   [junit4]   2> 656047 T2945 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard1_replica2
   [junit4]   2> 656048 T2919 oasc.SolrXMLCoresLocator.doPersist Persisted core descriptions to /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1396210347418/solr.xml
   [junit4]   2> 656048 T2886 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard2_replica1
   [junit4]   2> 656049 T2933 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard1_replica1
   [junit4]   2> 656048 T2919 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard2_replica2' using instanceDir: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1396210347418/solrj_collection_shard2_replica2
   [junit4]   2> 656050 T2933 oasc.SolrXMLCoresLocator.doPersist Persisted core descriptions to /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1396210348565/solr.xml
   [junit4]   2> 656050 T2919 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
   [junit4]   2> 656049 T2886 oasc.SolrXMLCoresLocator.doPersist Persisted core descriptions to /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1396210346084/solr.xml
   [junit4]   2> 656052 T2886 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard2_replica1' using instanceDir: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1396210346084/solrj_collection_shard2_replica1
   [junit4]   2> 656051 T2933 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard1_replica1' using instanceDir: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1396210348565/solrj_collection_shard1_replica1
   [junit4]   2> 656048 T2945 oasc.SolrXMLCoresLocator.doPersist Persisted core descriptions to /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1396210349715/solr.xml
   [junit4]   2> 656052 T2886 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
   [junit4]   2> 656053 T2933 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
   [junit4]   2> 656054 T2886 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 656053 T2945 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard1_replica2' using instanceDir: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1396210349715/solrj_collection_shard1_replica2
   [junit4]   2> 656054 T2886 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
   [junit4]   2> 656055 T2933 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 656052 T2919 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 656055 T2945 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
   [junit4]   2> 656056 T2919 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
   [junit4]   2> 656056 T2886 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 656056 T2933 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
   [junit4]   2> 656057 T2886 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1396210346084/solrj_collection_shard2_replica1/'
   [junit4]   2> 656057 T2945 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 656058 T2919 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 656059 T2945 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
   [junit4]   2> 656060 T2933 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 656060 T2919 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1396210347418/solrj_collection_shard2_replica2/'
   [junit4]   2> 656061 T2933 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1396210348565/solrj_collection_shard1_replica1/'
   [junit4]   2> 656061 T2945 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 656062 T2945 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1396210349715/solrj_collection_shard1_replica2/'
   [junit4]   2> 656101 T2886 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 656125 T2933 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 656125 T2919 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 656138 T2945 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 656174 T2886 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 656176 T2919 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 656181 T2886 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 656185 T2919 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 656204 T2933 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 656211 T2919 oass.IndexSchema.readSchema [solrj_collection_shard2_replica2] Schema name=test
   [junit4]   2> 656213 T2886 oass.IndexSchema.readSchema [solrj_collection_shard2_replica1] Schema name=test
   [junit4]   2> 656215 T2933 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 656224 T2933 oass.IndexSchema.readSchema [solrj_collection_shard1_replica1] Schema name=test
   [junit4]   2> 656230 T2945 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 656234 T2945 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 656260 T2945 oass.IndexSchema.readSchema [solrj_collection_shard1_replica2] Schema name=test
   [junit4]   2> 656759 T2919 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 656759 T2919 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 656763 T2919 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 656772 T2919 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 656775 T2919 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 656793 T2919 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 656799 T2945 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 656799 T2945 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 656800 T2945 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 656802 T2933 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 656803 T2933 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 656804 T2933 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 656801 T2919 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 656808 T2945 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 656808 T2933 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 656811 T2933 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 656812 T2945 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 656813 T2919 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 656815 T2919 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 656817 T2919 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 656820 T2919 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 656821 T2919 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 656824 T2919 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 656825 T2945 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 656826 T2933 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 656826 T2919 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 656827 T2886 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 656828 T2919 oasc.SolrCore.<init> [solrj_collection_shard2_replica2] Opening new SolrCore at ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1396210347418/solrj_collection_shard2_replica2/, dataDir=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1396210347418/solrj_collection_shard2_replica2/data/
   [junit4]   2> 656828 T2886 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 656828 T2919 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@ce179c5
   [junit4]   2> 656829 T2886 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 656830 T2919 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1396210347418/solrj_collection_shard2_replica2/data
   [junit4]   2> 656831 T2933 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 656831 T2945 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 656831 T2919 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1396210347418/solrj_collection_shard2_replica2/data/index/
   [junit4]   2> 656833 T2919 oasc.SolrCore.initIndex WARN [solrj_collection_shard2_replica2] Solr index directory './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1396210347418/solrj_collection_shard2_replica2/data/index' doesn't exist. Creating new index...
   [junit4]   2> 656835 T2933 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 656836 T2886 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 656836 T2933 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 656837 T2933 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 656837 T2945 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 656838 T2933 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 656839 T2933 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 656840 T2933 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 656840 T2886 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 656840 T2933 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 656841 T2945 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 656841 T2933 oasc.SolrCore.<init> [solrj_collection_shard1_replica1] Opening new SolrCore at ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1396210348565/solrj_collection_shard1_replica1/, dataDir=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1396210348565/solrj_collection_shard1_replica1/data/
   [junit4]   2> 656842 T2945 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 656843 T2945 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 656842 T2933 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@ce179c5
   [junit4]   2> 656844 T2945 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 656845 T2945 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 656845 T2945 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 656846 T2945 oasc.SolrCore.<init> [solrj_collection_shard1_replica2] Opening new SolrCore at ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1396210349715/solrj_collection_shard1_replica2/, dataDir=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1396210349715/solrj_collection_shard1_replica2/data/
   [junit4]   2> 656846 T2945 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@ce179c5
   [junit4]   2> 656856 T2919 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1396210347418/solrj_collection_shard2_replica2/data/index
   [junit4]   2> 656856 T2886 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 656857 T2933 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1396210348565/solrj_collection_shard1_replica1/data
   [junit4]   2> 656857 T2945 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1396210349715/solrj_collection_shard1_replica2/data
   [junit4]   2> 656858 T2933 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1396210348565/solrj_collection_shard1_replica1/data/index/
   [junit4]   2> 656858 T2945 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1396210349715/solrj_collection_shard1_replica2/data/index/
   [junit4]   2> 656859 T2933 oasc.SolrCore.initIndex WARN [solrj_collection_shard1_replica1] Solr index directory './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1396210348565/solrj_collection_shard1_replica1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 656859 T2945 oasc.SolrCore.initIndex WARN [solrj_collection_shard1_replica2] Solr index directory './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1396210349715/solrj_collection_shard1_replica2/data/index' doesn't exist. Creating new index...
   [junit4]   2> 656860 T2933 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1396210348565/solrj_collection_shard1_replica1/data/index
   [junit4]   2> 656861 T2933 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=10, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.29804203210410907]
   [junit4]   2> 656861 T2945 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1396210349715/solrj_collection_shard1_replica2/data/index
   [junit4]   2> 656862 T2933 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@d385f432 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@c441e5c5),segFN=segments_1,generation=1}
   [junit4]   2> 656857 T2919 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=10, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.29804203210410907]
   [junit4]   2> 656863 T2933 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 656862 T2945 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=10, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.29804203210410907]
   [junit4]   2> 656864 T2919 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@ce39b47e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@c118706b),segFN=segments_1,generation=1}
   [junit4]   2> 656866 T2919 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 656863 T2886 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 656866 T2945 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@2f82a3d9 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@b85d4833),segFN=segments_1,generation=1}
   [junit4]   2> 656867 T2933 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 656868 T2945 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 656869 T2933 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 656870 T2933 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 656870 T2933 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 656871 T2933 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 656872 T2933 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 656873 T2933 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 656874 T2945 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 656874 T2933 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 656874 T2945 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 656875 T2933 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 656875 T2945 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 656876 T2933 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 656870 T2919 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 656876 T2945 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 656877 T2933 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 656878 T2933 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 656878 T2945 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 656877 T2919 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 656879 T2945 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 656879 T2933 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 656880 T2919 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 656881 T2933 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 656880 T2945 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 656881 T2919 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 656882 T2933 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 656882 T2945 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 656883 T2919 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 656884 T2945 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 656884 T2933 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 656885 T2919 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 656885 T2945 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 656876 T2886 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 656887 T2945 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 656889 T2945 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 656887 T2933 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 656886 T2919 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 656890 T2919 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 656889 T2945 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 656888 T2886 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 656892 T2945 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 656892 T2933 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 656891 T2919 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 656894 T2933 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 656893 T2945 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 656893 T2886 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 656895 T2933 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=29, maxMergeAtOnceExplicit=31, maxMergedSegmentMB=55.576171875, floorSegmentMB=0.3466796875, forceMergeDeletesPctAllowed=27.268316204003472, segmentsPerTier=34.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6162944771818158
   [junit4]   2> 656895 T2945 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 656894 T2919 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 656898 T2919 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 656899 T2919 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 656897 T2933 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@d385f432 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@c441e5c5),segFN=segments_1,generation=1}
   [junit4]   2> 656898 T2945 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 656896 T2886 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 656901 T2933 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 656902 T2886 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 656903 T2886 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 656904 T2933 oass.SolrIndexSearcher.<init> Opening Searcher@84c044f3[solrj_collection_shard1_replica1] main
   [junit4]   2> 656904 T2945 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 656904 T2886 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 656905 T2945 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 656906 T2886 oasc.SolrCore.<init> [solrj_collection_shard2_replica1] Opening new SolrCore at ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1396210346084/solrj_collection_shard2_replica1/, dataDir=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1396210346084/solrj_collection_shard2_replica1/data/
   [junit4]   2> 656907 T2945 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=29, maxMergeAtOnceExplicit=31, maxMergedSegmentMB=55.576171875, floorSegmentMB=0.3466796875, forceMergeDeletesPctAllowed=27.268316204003472, segmentsPerTier=34.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6162944771818158
   [junit4]   2> 656907 T2886 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@ce179c5
   [junit4]   2> 656899 T2919 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 656908 T2945 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@2f82a3d9 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@b85d4833),segFN=segments_1,generation=1}
   [junit4]   2> 656909 T2886 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1396210346084/solrj_collection_shard2_replica1/data
   [junit4]   2> 656909 T2945 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 656910 T2886 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1396210346084/solrj_collection_shard2_replica1/data/index/
   [junit4]   2> 656910 T2945 oass.SolrIndexSearcher.<init> Opening Searcher@95f591b3[solrj_collection_shard1_replica2] main
   [junit4]   2> 656911 T2886 oasc.SolrCore.initIndex WARN [solrj_collection_shard2_replica1] Solr index directory './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1396210346084/solrj_collection_shard2_replica1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 656909 T2919 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 656912 T2886 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1396210346084/solrj_collection_shard2_replica1/data/index
   [junit4]   2> 656912 T2919 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 656913 T2886 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=10, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.29804203210410907]
   [junit4]   2> 656913 T2959 oasc.SolrCore.registerSearcher [solrj_collection_shard1_replica2] Registered new searcher Searcher@95f591b3[solrj_collection_shard1_replica2] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 656914 T2886 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@d70061c7 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@160815f),segFN=segments_1,generation=1}
   [junit4]   2> 656913 T2919 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 656914 T2886 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 656915 T2945 oasc.CoreContainer.registerCore registering core: solrj_collection_shard1_replica2
   [junit4]   2> 656916 T2945 oasc.ZkController.register Register replica - core:solrj_collection_shard1_replica2 address:http://127.0.0.1:39330/cez/fi collection:solrj_collection shard:shard1
   [junit4]   2> 656916 T2919 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 656917 T2945 oascc.SolrZkClient.makePath makePath: /collections/solrj_collection/leader_elect/shard1/election
   [junit4]   2> 656918 T2886 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 656919 T2886 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 656920 T2886 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 656920 T2919 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 656907 T2958 oasc.SolrCore.registerSearcher [solrj_collection_shard1_replica1] Registered new searcher Searcher@84c044f3[solrj_collection_shard1_replica1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 656921 T2919 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 656921 T2886 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 656922 T2919 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=29, maxMergeAtOnceExplicit=31, maxMergedSegmentMB=55.576171875, floorSegmentMB=0.3466796875, forceMergeDeletesPctAllowed=27.268316204003472, segmentsPerTier=34.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6162944771818158
   [junit4]   2> 656922 T2886 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 656923 T2945 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 656923 T2886 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 656923 T2933 oasc.CoreContainer.registerCore registering core: solrj_collection_shard1_replica1
   [junit4]   2> 656923 T2919 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@ce39b47e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@c118706b),segFN=segments_1,generation=1}
   [junit4]   2> 656925 T2886 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 656925 T2933 oasc.ZkController.register Register replica - core:solrj_collection_shard1_replica1 address:http://127.0.0.1:37752/cez/fi collection:solrj_collection shard:shard1
   [junit4]   2> 656925 T2892 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 656926 T2919 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 656926 T2945 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=6.8236186E14ms
   [junit4]   2> 656928 T2945 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Was waiting for replicas to come up, but they are taking too long - assuming they won't come back till later
   [junit4]   2> 656928 T2893 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 656927 T2892 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 656926 T2886 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 656929 T2892 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 656930 T2886 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 656928 T2945 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> 656931 T2892 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> ASYNC  NEW_CORE C4691 name=solrj_collection_shard1_replica2 org.apache.solr.core.SolrCore@8fcce809 url=http://127.0.0.1:39330/cez/fi/solrj_collection_shard1_replica2 node=127.0.0.1:39330_cez%2Ffi C4691_STATE=coll:solrj_collection core:solrj_collection_shard1_replica2 props:{state=down, base_url=http://127.0.0.1:39330/cez/fi, core=solrj_collection_shard1_replica2, node_name=127.0.0.1:39330_cez%2Ffi}
   [junit4]   2> 656931 T2945 C4691 P39330 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:39330/cez/fi/solrj_collection_shard1_replica2/
   [junit4]   2> 656931 T2886 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 656927 T2919 oass.SolrIndexSearcher.<init> Opening Searcher@52fe27a2[solrj_collection_shard2_replica2] main
   [junit4]   2> 656932 T2945 C4691 P39330 oasu.PeerSync.sync PeerSync: core=solrj_collection_shard1_replica2 url=http://127.0.0.1:39330/cez/fi START replicas=[http://127.0.0.1:37752/cez/fi/solrj_collection_shard1_replica1/] nUpdates=100
   [junit4]   2> 656934 T2892 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 656935 T2952 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 656934 T2910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 656933 T2886 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 656935 T2945 C4691 P39330 oasu.PeerSync.sync PeerSync: core=solrj_collection_shard1_replica2 url=http://127.0.0.1:39330/cez/fi DONE.  We have no versions.  sync failed.
   [junit4]   2> 656937 T2945 C4691 P39330 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 656934 T2924 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 656934 T2938 oascc.ZkStateReader$2.process A cluster state change: Watch

[...truncated too long message...]

s=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
   [junit4]   2> 847316 T2866 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 847317 T2866 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 847317 T2866 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 847318 T2952 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 847320 T2866 oasc.SolrCore.closeSearcher [solrj_collection_shard1_replica2] Closing main searcher on request.
   [junit4]   2> 847321 T2866 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 847332 T2866 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1396210349715/solrj_collection_shard1_replica2/data [CachedDir<<refCount=0;path=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1396210349715/solrj_collection_shard1_replica2/data;done=false>>]
   [junit4]   2> 847332 T2866 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1396210349715/solrj_collection_shard1_replica2/data
   [junit4]   2> 847334 T2866 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1396210349715/solrj_collection_shard1_replica2/data/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1396210349715/solrj_collection_shard1_replica2/data/index;done=false>>]
   [junit4]   2> 847334 T2866 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1396210349715/solrj_collection_shard1_replica2/data/index
   [junit4]   2> 847335 T2866 oasc.SolrCore.close [conf1_shard4_replica1]  CLOSING SolrCore org.apache.solr.core.SolrCore@b63b7ee2
   [junit4]   2> 847350 T2866 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
   [junit4]   2> 847351 T2866 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 847352 T2866 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 847353 T2866 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 847359 T2866 oasc.SolrCore.closeSearcher [conf1_shard4_replica1] Closing main searcher on request.
   [junit4]   2> 847363 T2866 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 847365 T2866 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1396210349715/conf1_shard4_replica1/data/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1396210349715/conf1_shard4_replica1/data/index;done=false>>]
   [junit4]   2> 847365 T2866 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1396210349715/conf1_shard4_replica1/data/index
   [junit4]   2> 847366 T2866 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1396210349715/conf1_shard4_replica1/data [CachedDir<<refCount=0;path=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1396210349715/conf1_shard4_replica1/data;done=false>>]
   [junit4]   2> 847367 T2866 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1396210349715/conf1_shard4_replica1/data
   [junit4]   2> 847376 T2952 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 847403 T2866 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/cez/fi,null}
   [junit4]   2> 847459 T2866 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
   [junit4]   2> 847466 T2866 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:55386 55386
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=CollectionsAPIDistributedZkTest -Dtests.method=testDistribSearch -Dtests.seed=D19CFC9A78C27D96 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=lt_LT -Dtests.timezone=Africa/Dar_es_Salaam -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR    198s J1 | CollectionsAPIDistributedZkTest.testDistribSearch <<<
   [junit4]    > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: deletealias the collection time out:180s
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([D19CFC9A78C27D96:507A72820F9D1DAA]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:495)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:199)
   [junit4]    > 	at org.apache.solr.client.solrj.request.CollectionAdminRequest.process(CollectionAdminRequest.java:272)
   [junit4]    > 	at org.apache.solr.client.solrj.request.CollectionAdminRequest.deleteAlias(CollectionAdminRequest.java:404)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testSolrJAPICalls(CollectionsAPIDistributedZkTest.java:304)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:199)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:870)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:853)
   [junit4]   2> 847709 T2866 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> 197784 T2865 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 4 leaked thread(s).
   [junit4]   2> 848218 T2892 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 848218 T2892 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 848368 T2910 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 848700 T2938 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> NOTE: test params are: codec=Lucene42, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=lt_LT, timezone=Africa/Dar_es_Salaam
   [junit4]   2> NOTE: Linux 3.8.0-37-generic x86/IBM Corporation 1.7.0 (32-bit)/cpus=8,threads=1,free=197466912,total=428015616
   [junit4]   2> NOTE: All tests run in this JVM: [TestReload, ChaosMonkeySafeLeaderTest, TestCSVLoader, TestCustomSort, TestRecovery, TestCloudManagedSchema, SolrRequestParserTest, FieldAnalysisRequestHandlerTest, PrimUtilsTest, SuggesterWFSTTest, DateFieldTest, TestQueryTypes, StatsComponentTest, TestCopyFieldCollectionResource, ChaosMonkeyNothingIsSafeTest, TestFuzzyAnalyzedSuggestions, AddBlockUpdateTest, SolrCoreCheckLockOnStartupTest, DirectSolrConnectionTest, DirectSolrSpellCheckerTest, CachingDirectoryFactoryTest, CoreContainerCoreInitFailuresTest, JSONWriterTest, TestDistributedMissingSort, TestZkChroot, TestCollationFieldDocValues, TestImplicitCoreProperties, BasicZkTest, TestPhraseSuggestions, HdfsDirectoryTest, TestSerializedLuceneMatchVersion, ConvertedLegacyTest, TestLazyCores, SpellingQueryConverterTest, RAMDirectoryFactoryTest, TestRTGBase, TestFiltering, HdfsLockFactoryTest, ZkSolrClientTest, SolrCmdDistributorTest, TestFastWriter, TermVectorComponentDistributedTest, SpellCheckComponentTest, IndexSchemaRuntimeFieldTest, SignatureUpdateProcessorFactoryTest, TestSolrQueryParserDefaultOperatorResource, TestFreeTextSuggestions, BasicDistributedZk2Test, TestDFRSimilarityFactory, TestHashPartitioner, TestFastLRUCache, TestSolrCoreProperties, TestMultiCoreConfBootstrap, DebugComponentTest, TestUtils, TestAddFieldRealTimeGet, InfoHandlerTest, CollectionsAPIDistributedZkTest]
   [junit4] Completed on J1 in 198.83s, 1 test, 1 error <<< FAILURES!

[...truncated 673 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/build.xml:459: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/build.xml:452: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/common-build.xml:505: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/lucene/common-build.xml:1278: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/lucene/common-build.xml:911: There were test failures: 365 suites, 1570 tests, 1 error, 35 ignored (7 assumptions)

Total time: 61 minutes 39 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/ibm-j9-jdk7 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-4.7-Linux (32bit/jdk1.7.0_51) - Build # 56 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.7-Linux/56/
Java: 32bit/jdk1.7.0_51 -client -XX:+UseSerialGC

1 tests failed.
REGRESSION:  org.apache.solr.cloud.ShardSplitTest.testDistribSearch

Error Message:
Timeout occured while waiting response from server at: https://127.0.0.1:33450/_g

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:33450/_g
	at __randomizedtesting.SeedInfo.seed([A43B796BBD0003A9:25DDF773CA5F6395]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:503)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:199)
	at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:536)
	at org.apache.solr.cloud.ShardSplitTest.incompleteOrOverlappingCustomRangeTest(ShardSplitTest.java:159)
	at org.apache.solr.cloud.ShardSplitTest.doTest(ShardSplitTest.java:111)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:870)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:744)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
	at sun.security.ssl.InputRecord.read(InputRecord.java:480)
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927)
	at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:884)
	at sun.security.ssl.AppInputStream.read(AppInputStream.java:102)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:260)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:271)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:682)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:486)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:395)
	... 45 more




Build Log:
[...truncated 10534 lines...]
   [junit4] Suite: org.apache.solr.cloud.ShardSplitTest
   [junit4]   2> 15250 T44 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
   [junit4]   2> 15250 T44 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /_g/
   [junit4]   2> 15254 T44 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1/./solrtest-ShardSplitTest-1396229261646
   [junit4]   2> 15257 T44 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 15259 T45 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 15359 T44 oasc.ZkTestServer.run start zk server on port:44935
   [junit4]   2> 15418 T44 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 15819 T51 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@fcea2a name:ZooKeeperConnection Watcher:127.0.0.1:44935 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 15820 T44 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 15822 T44 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 15850 T44 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 15854 T53 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1bf9583 name:ZooKeeperConnection Watcher:127.0.0.1:44935/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 15854 T44 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 15860 T44 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 15865 T44 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 15870 T44 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 15879 T44 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 15884 T44 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 15896 T44 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 15903 T44 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 15904 T44 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 15909 T44 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 15909 T44 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 15914 T44 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 15915 T44 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 15920 T44 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 15921 T44 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 15924 T44 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 15925 T44 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 15929 T44 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 15930 T44 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 15934 T44 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 15934 T44 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 15938 T44 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 15939 T44 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 15944 T44 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 15945 T44 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 15949 T44 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 15949 T44 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 15959 T44 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 15962 T55 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e71cec name:ZooKeeperConnection Watcher:127.0.0.1:44935/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 15963 T44 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 16198 T44 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 16474 T44 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 16495 T44 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:35847
   [junit4]   2> 16536 T44 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 16537 T44 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 16537 T44 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.ShardSplitTest-controljetty-1396229262356
   [junit4]   2> 16538 T44 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ShardSplitTest-controljetty-1396229262356/'
   [junit4]   2> 16560 T44 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-controljetty-1396229262356/solr.xml
   [junit4]   2> 16613 T44 oasc.CoreContainer.<init> New CoreContainer 1987332
   [junit4]   2> 16613 T44 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.ShardSplitTest-controljetty-1396229262356/]
   [junit4]   2> 16625 T44 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 16625 T44 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 16625 T44 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 16626 T44 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 16626 T44 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 16626 T44 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 16627 T44 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 16627 T44 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 16627 T44 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 16630 T44 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 16630 T44 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 16630 T44 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 16631 T44 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:44935/solr
   [junit4]   2> 16640 T44 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 16641 T44 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 16644 T67 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@906ae name:ZooKeeperConnection Watcher:127.0.0.1:44935 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 16644 T44 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 16651 T44 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 16653 T69 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19292b0 name:ZooKeeperConnection Watcher:127.0.0.1:44935/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 16653 T44 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 16658 T44 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 16667 T44 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 16677 T44 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 16680 T44 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35847__g
   [junit4]   2> 16682 T44 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:35847__g
   [junit4]   2> 16687 T44 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 16690 T44 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 16700 T44 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:35847__g
   [junit4]   2> 16700 T44 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 16704 T44 oasc.Overseer.start Overseer (id=91503280897982468-127.0.0.1:35847__g-n_0000000000) starting
   [junit4]   2> 16712 T44 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 16724 T44 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 16724 T71 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 16726 T71 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
   [junit4]   2> 16731 T44 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 16733 T44 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 16741 T70 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 16753 T72 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
   [junit4]   2> 16753 T72 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 16757 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 16757 T72 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 16760 T70 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 16762 T70 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:35847/_g",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:35847__g",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 16762 T70 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
   [junit4]   2> 16768 T70 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 16772 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 16778 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 17758 T72 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 17758 T72 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.ShardSplitTest-controljetty-1396229262356/collection1
   [junit4]   2> 17759 T72 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
   [junit4]   2> 17760 T72 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 17760 T72 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 17761 T72 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 17762 T72 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ShardSplitTest-controljetty-1396229262356/collection1/'
   [junit4]   2> 17763 T72 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-controljetty-1396229262356/collection1/lib/classes/' to classloader
   [junit4]   2> 17763 T72 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-controljetty-1396229262356/collection1/lib/README' to classloader
   [junit4]   2> 17817 T72 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 17871 T72 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 17872 T72 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 17881 T72 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 18320 T72 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 18323 T72 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 18325 T72 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 18331 T72 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 18367 T72 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 18367 T72 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.ShardSplitTest-controljetty-1396229262356/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1396229261646/control/data/
   [junit4]   2> 18368 T72 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@fe6823
   [junit4]   2> 18372 T72 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1396229261646/control/data
   [junit4]   2> 18372 T72 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1396229261646/control/data/index/
   [junit4]   2> 18372 T72 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1396229261646/control/data/index' doesn't exist. Creating new index...
   [junit4]   2> 18373 T72 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1396229261646/control/data/index
   [junit4]   2> 18373 T72 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=36, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8245680680725896]
   [junit4]   2> 18374 T72 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@1db19c9 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@14a1247),segFN=segments_1,generation=1}
   [junit4]   2> 18375 T72 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 18382 T72 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 18395 T72 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 18396 T72 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 18396 T72 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 18396 T72 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 18396 T72 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 18397 T72 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 18397 T72 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 18397 T72 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 18399 T72 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 18400 T72 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 18400 T72 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 18401 T72 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 18402 T72 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 18402 T72 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 18406 T72 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 18408 T72 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 18411 T72 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 18411 T72 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 18412 T72 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=46, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8228753845148041]
   [junit4]   2> 18413 T72 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@1db19c9 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@14a1247),segFN=segments_1,generation=1}
   [junit4]   2> 18413 T72 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 18414 T72 oass.SolrIndexSearcher.<init> Opening Searcher@123be1a[collection1] main
   [junit4]   2> 18420 T73 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@123be1a[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 18424 T72 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 18425 T44 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 18425 T44 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 18429 T76 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:35847/_g collection:control_collection shard:shard1
   [junit4]   2> 18437 T76 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 18444 T44 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 18446 T78 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@902554 name:ZooKeeperConnection Watcher:127.0.0.1:44935/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 18446 T44 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 18446 T76 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 18448 T44 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 18449 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 18450 T76 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 18451 T76 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C22 name=collection1 org.apache.solr.core.SolrCore@1d3768f url=https://127.0.0.1:35847/_g/collection1 node=127.0.0.1:35847__g C22_STATE=coll:control_collection core:collection1 props:{state=down, base_url=https://127.0.0.1:35847/_g, core=collection1, node_name=127.0.0.1:35847__g}
   [junit4]   2> 18451 T76 C22 P35847 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:35847/_g/collection1/
   [junit4]   2> 18452 T70 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 18452 T76 C22 P35847 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 18452 T76 C22 P35847 oasc.SyncStrategy.syncToMe https://127.0.0.1:35847/_g/collection1/ has no replicas
   [junit4]   2> 18452 T44 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 18453 T76 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:35847/_g/collection1/ shard1
   [junit4]   2> 18453 T76 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 18457 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 18458 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 18458 T78 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 18509 T70 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 18514 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 18547 T44 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
   [junit4]   2> 18548 T44 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 18557 T44 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 18560 T44 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:33450
   [junit4]   2> 18561 T44 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 18562 T44 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 18563 T44 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.ShardSplitTest-jetty1-1396229264844
   [junit4]   2> 18564 T44 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ShardSplitTest-jetty1-1396229264844/'
   [junit4]   2> 18577 T44 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty1-1396229264844/solr.xml
   [junit4]   2> 18617 T78 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 18617 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 18629 T44 oasc.CoreContainer.<init> New CoreContainer 2342800
   [junit4]   2> 18629 T44 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.ShardSplitTest-jetty1-1396229264844/]
   [junit4]   2> 18631 T44 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 18631 T44 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 18631 T44 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 18631 T44 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 18632 T44 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 18632 T44 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 18632 T44 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 18633 T44 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 18633 T44 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 18636 T44 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 18636 T44 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 18636 T44 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 18637 T44 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:44935/solr
   [junit4]   2> 18637 T44 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 18638 T44 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 18639 T89 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@22a7e1 name:ZooKeeperConnection Watcher:127.0.0.1:44935 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 18640 T44 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 18642 T44 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 18644 T91 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@18bbd6f name:ZooKeeperConnection Watcher:127.0.0.1:44935/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 18644 T44 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 18649 T44 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 18658 T76 oasc.ZkController.register We are https://127.0.0.1:35847/_g/collection1/ and leader is https://127.0.0.1:35847/_g/collection1/
   [junit4]   2> 18659 T76 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:35847/_g
   [junit4]   2> 18659 T76 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 18659 T76 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
   [junit4]   2> 18659 T76 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 18661 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 18661 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 18661 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 18661 T76 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 18663 T70 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 18664 T70 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:35847/_g",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:35847__g",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 18667 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 18774 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 18774 T91 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 18774 T78 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 19653 T44 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33450__g
   [junit4]   2> 19655 T44 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:33450__g
   [junit4]   2> 19658 T69 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 19659 T78 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 19659 T91 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 19668 T92 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 19668 T92 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 19670 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 19670 T92 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 19670 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 19671 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 19672 T70 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 19673 T70 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:33450/_g",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:33450__g",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 19674 T70 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
   [junit4]   2> 19674 T70 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 19679 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 19855 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 19856 T91 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 19856 T78 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 20671 T92 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 20671 T92 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.ShardSplitTest-jetty1-1396229264844/collection1
   [junit4]   2> 20671 T92 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 20672 T92 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 20672 T92 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 20674 T92 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 20674 T92 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ShardSplitTest-jetty1-1396229264844/collection1/'
   [junit4]   2> 20674 T92 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty1-1396229264844/collection1/lib/classes/' to classloader
   [junit4]   2> 20675 T92 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty1-1396229264844/collection1/lib/README' to classloader
   [junit4]   2> 20725 T92 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 20783 T92 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 20785 T92 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 20798 T92 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 21328 T92 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 21331 T92 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 21333 T92 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 21339 T92 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 21366 T92 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 21367 T92 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.ShardSplitTest-jetty1-1396229264844/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1396229261646/jetty1/
   [junit4]   2> 21367 T92 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@fe6823
   [junit4]   2> 21368 T92 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1396229261646/jetty1
   [junit4]   2> 21368 T92 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1396229261646/jetty1/index/
   [junit4]   2> 21369 T92 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1396229261646/jetty1/index' doesn't exist. Creating new index...
   [junit4]   2> 21369 T92 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1396229261646/jetty1/index
   [junit4]   2> 21369 T92 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=36, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8245680680725896]
   [junit4]   2> 21370 T92 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@3bf6a7 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1932006),segFN=segments_1,generation=1}
   [junit4]   2> 21370 T92 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 21377 T92 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 21378 T92 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 21378 T92 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 21378 T92 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 21378 T92 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 21379 T92 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 21379 T92 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 21379 T92 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 21379 T92 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 21381 T92 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 21382 T92 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 21382 T92 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 21382 T92 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 21383 T92 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 21384 T92 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 21384 T92 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 21387 T92 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 21392 T92 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 21392 T92 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 21394 T92 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=46, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8228753845148041]
   [junit4]   2> 21394 T92 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@3bf6a7 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1932006),segFN=segments_1,generation=1}
   [junit4]   2> 21394 T92 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 21395 T92 oass.SolrIndexSearcher.<init> Opening Searcher@13eb044[collection1] main
   [junit4]   2> 21403 T93 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@13eb044[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 21407 T92 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 21408 T44 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 21408 T44 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 21408 T96 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:33450/_g collection:collection1 shard:shard1
   [junit4]   2> 21410 T96 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 21417 T96 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 21420 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 21420 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 21420 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 21420 T96 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 21421 T96 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C23 name=collection1 org.apache.solr.core.SolrCore@17d3a97 url=https://127.0.0.1:33450/_g/collection1 node=127.0.0.1:33450__g C23_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:33450/_g, core=collection1, node_name=127.0.0.1:33450__g}
   [junit4]   2> 21421 T96 C23 P33450 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:33450/_g/collection1/
   [junit4]   2> 21422 T96 C23 P33450 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 21422 T70 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 21422 T96 C23 P33450 oasc.SyncStrategy.syncToMe https://127.0.0.1:33450/_g/collection1/ has no replicas
   [junit4]   2> 21423 T96 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:33450/_g/collection1/ shard1
   [junit4]   2> 21423 T96 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 21426 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 21428 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 21428 T91 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 21428 T78 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 21435 T70 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 21441 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 21496 T44 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
   [junit4]   2> 21499 T44 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 21503 T44 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 21509 T44 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:38303
   [junit4]   2> 21513 T44 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 21513 T44 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 21514 T44 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1396229267802
   [junit4]   2> 21514 T44 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1396229267802/'
   [junit4]   2> 21533 T44 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1396229267802/solr.xml
   [junit4]   2> 21544 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 21544 T91 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 21544 T78 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 21636 T96 oasc.ZkController.register We are https://127.0.0.1:33450/_g/collection1/ and leader is https://127.0.0.1:33450/_g/collection1/
   [junit4]   2> 21637 T96 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:33450/_g
   [junit4]   2> 21637 T96 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 21637 T96 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
   [junit4]   2> 21637 T96 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 21638 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 21639 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 21639 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 21639 T96 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 21640 T70 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 21641 T70 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:33450/_g",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:33450__g",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 21645 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 21681 T44 oasc.CoreContainer.<init> New CoreContainer 15228697
   [junit4]   2> 21682 T44 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1396229267802/]
   [junit4]   2> 21683 T44 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 21683 T44 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 21684 T44 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 21684 T44 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 21684 T44 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 21685 T44 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 21685 T44 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 21685 T44 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 21686 T44 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 21689 T44 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 21689 T44 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 21689 T44 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 21690 T44 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:44935/solr
   [junit4]   2> 21690 T44 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 21691 T44 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 21692 T107 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1acb490 name:ZooKeeperConnection Watcher:127.0.0.1:44935 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 21693 T44 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 21695 T44 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 21696 T109 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@937938 name:ZooKeeperConnection Watcher:127.0.0.1:44935/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 21697 T44 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 21701 T44 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 21748 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 21748 T91 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 21748 T78 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 21748 T109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 22704 T44 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38303__g
   [junit4]   2> 22705 T44 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38303__g
   [junit4]   2> 22708 T91 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 22708 T69 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 22708 T109 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 22708 T78 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 22718 T110 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 22718 T110 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 22719 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 22719 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 22719 T110 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 22720 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 22721 T70 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 22722 T70 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:38303/_g",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:38303__g",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 22723 T70 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
   [junit4]   2> 22723 T70 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 22726 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 22828 T78 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 22829 T109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 22829 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 22828 T91 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 23720 T110 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 23720 T110 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1396229267802/collection1
   [junit4]   2> 23721 T110 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 23722 T110 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 23722 T110 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 23723 T110 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 23723 T110 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1396229267802/collection1/'
   [junit4]   2> 23727 T110 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty2-1396229267802/collection1/lib/classes/' to classloader
   [junit4]   2> 23727 T110 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty2-1396229267802/collection1/lib/README' to classloader
   [junit4]   2> 23790 T110 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 23869 T110 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 23871 T110 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 23885 T110 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 24310 T110 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 24315 T110 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 24317 T110 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 24321 T110 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 24345 T110 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 24346 T110 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1396229267802/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1396229261646/jetty2/
   [junit4]   2> 24346 T110 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@fe6823
   [junit4]   2> 24347 T110 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1396229261646/jetty2
   [junit4]   2> 24347 T110 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1396229261646/jetty2/index/
   [junit4]   2> 24347 T110 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1396229261646/jetty2/index' doesn't exist. Creating new index...
   [junit4]   2> 24348 T110 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1396229261646/jetty2/index
   [junit4]   2> 24348 T110 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=36, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8245680680725896]
   [junit4]   2> 24349 T110 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@4fb1d4 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1644765),segFN=segments_1,generation=1}
   [junit4]   2> 24349 T110 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 24353 T110 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 24354 T110 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 24354 T110 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 24354 T110 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 24355 T110 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 24355 T110 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 24355 T110 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 24355 T110 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 24356 T110 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 24356 T110 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 24357 T110 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 24357 T110 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 24358 T110 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 24359 T110 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 24359 T110 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 24360 T110 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 24362 T110 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 24365 T110 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 24365 T110 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 24366 T110 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=46, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8228753845148041]
   [junit4]   2> 24367 T110 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@4fb1d4 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1644765),segFN=segments_1,generation=1}
   [junit4]   2> 24367 T110 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 24367 T110 oass.SolrIndexSearcher.<init> Opening Searcher@1cde778[collection1] main
   [junit4]   2> 24373 T111 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1cde778[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 24379 T110 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 24380 T44 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 24381 T44 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 24380 T114 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:38303/_g collection:collection1 shard:shard2
   [junit4]   2> 24382 T114 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
   [junit4]   2> 24399 T114 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
   [junit4]   2> 24401 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 24402 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 24402 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 24402 T114 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 24402 T114 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C24 name=collection1 org.apache.solr.core.SolrCore@15276c4 url=https://127.0.0.1:38303/_g/collection1 node=127.0.0.1:38303__g C24_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:38303/_g, core=collection1, node_name=127.0.0.1:38303__g}
   [junit4]   2> 24403 T114 C24 P38303 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:38303/_g/collection1/
   [junit4]   2> 24403 T114 C24 P38303 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 24403 T114 C24 P38303 oasc.SyncStrategy.syncToMe https://127.0.0.1:38303/_g/collection1/ has no replicas
   [junit4]   2> 24404 T114 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:38303/_g/collection1/ shard2
   [junit4]   2> 24404 T114 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
   [junit4]   2> 24405 T70 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 24409 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 24415 T91 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 24416 T78 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 24416 T109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 24415 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 24420 T70 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 24423 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 24468 T44 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
   [junit4]   2> 24469 T44 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 24472 T44 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 24475 T44 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:39212
   [junit4]   2> 24476 T44 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 24477 T44 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 24477 T44 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.ShardSplitTest-jetty3-1396229270773
   [junit4]   2> 24477 T44 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ShardSplitTest-jetty3-1396229270773/'
   [junit4]   2> 24490 T44 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty3-1396229270773/solr.xml
   [junit4]   2> 24526 T78 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 24526 T91 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 24526 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 24526 T109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 24544 T44 oasc.CoreContainer.<init> New CoreContainer 27149162
   [junit4]   2> 24544 T44 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.ShardSplitTest-jetty3-1396229270773/]
   [junit4]   2> 24548 T44 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 24548 T44 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 24548 T44 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 24548 T44 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 24549 T44 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 24549 T44 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 24549 T44 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 24550 T44 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 24550 T44 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 24552 T44 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 24553 T44 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 24553 T44 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 24553 T44 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:44935/solr
   [junit4]   2> 24554 T44 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 24554 T44 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 24556 T125 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5239a name:ZooKeeperConnection Watcher:127.0.0.1:44935 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 24556 T44 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 24558 T44 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 24560 T127 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2b7d5a name:ZooKeeperConnection Watcher:127.0.0.1:44935/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 24560 T44 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 24564 T44 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 24568 T114 oasc.ZkController.register We are https://127.0.0.1:38303/_g/collection1/ and leader is https://127.0.0.1:38303/_g/collection1/
   [junit4]   2> 24569 T114 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:38303/_g
   [junit4]   2> 24569 T114 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 24569 T114 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
   [junit4]   2> 24569 T114 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 24572 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 24572 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 24572 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 24573 T114 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 24574 T70 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 24575 T70 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:38303/_g",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:38303__g",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node2"}
   [junit4]   2> 24578 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 24681 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 24682 T109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 24682 T127 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 24681 T91 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 24681 T78 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 25567 T44 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39212__g
   [junit4]   2> 25569 T44 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:39212__g
   [junit4]   2> 25572 T109 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 25572 T91 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 25573 T78 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 25573 T69 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 25573 T127 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 25580 T128 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 25580 T128 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 25581 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 25581 T128 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 25582 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 25582 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 25584 T70 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 25584 T70 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:39212/_g",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:39212__g",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 25585 T70 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
   [junit4]   2> 25585 T70 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 25588 T69 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 25691 T78 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 25691 T127 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 25691 T91 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 25691 T69 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 25691 T109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 26585 T128 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 26585 T128 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.ShardSplitTest-jetty3-1396229270773/collection1
   [junit4]   2> 26585 T128 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 26586 T128 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 26586 T128 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 26588 T128 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 26588 T128 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ShardSplitTest-jetty3-1396229270773/collection1/'
   [junit4]   2> 26589 T128 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty3-1396229270773/collection1/lib/classes/' to classloader
   [junit4]   2> 26589 T128 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty3-1396229270773/collection1/lib/README' to classloader
   [junit4]   2> 26662 T128 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 26747 T128 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 26787 T128 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 26799 T128 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 27336 T128 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 27339 T128 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 27341 T128 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 27345 T128 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 27367 T128 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 27367 T128 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.ShardSplitTest-jetty3-1396229270773/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1396229261646/jetty3/
   [junit4]   2> 27368 T128 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@fe6823
   [junit4]   2> 27368 T128 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.S

[...truncated too long message...]

137)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> ASYNC  NEW_CORE C41 name=collection1 org.apache.solr.core.SolrCore@1f66867 url=https://127.0.0.1:60685/_g/collection1 node=127.0.0.1:60685__g C41_STATE=coll:collection1 core:collection1 props:{state=active, base_url=https://127.0.0.1:60685/_g, core=collection1, node_name=127.0.0.1:60685__g}
   [junit4]   2> 340778 T146 C41 P60685 oasc.SyncStrategy.sync WARN Closed, skipping sync up.
   [junit4]   2> 340778 T146 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 340779 T146 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:60685/_g/collection1/ shard2
   [junit4]   2> 340779 T146 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@1f66867
   [junit4]   2> 340802 T146 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
   [junit4]   2> 340803 T146 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 340803 T146 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 340803 T146 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 340804 T146 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 340805 T146 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 340805 T146 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ShardSplitTest-1396229261646/jetty4 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ShardSplitTest-1396229261646/jetty4;done=false>>]
   [junit4]   2> 340805 T146 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ShardSplitTest-1396229261646/jetty4
   [junit4]   2> 340805 T146 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ShardSplitTest-1396229261646/jetty4/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ShardSplitTest-1396229261646/jetty4/index;done=false>>]
   [junit4]   2> 340806 T146 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ShardSplitTest-1396229261646/jetty4/index
   [junit4]   2> 340806 T146 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
   [junit4]   2> 340806 T146 oasc.SolrException.log ERROR There was a problem trying to register as the leader:org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections
   [junit4]   2> 		at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 		at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 		at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:206)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:203)
   [junit4]   2> 		at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:203)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:414)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:383)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:370)
   [junit4]   2> 		at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:113)
   [junit4]   2> 		at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:274)
   [junit4]   2> 		at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:164)
   [junit4]   2> 		at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:108)
   [junit4]   2> 		at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 		at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
   [junit4]   2> 		at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 		at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 	
   [junit4]   2> 340807 T146 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard2/election/91503280897982477-core_node4-n_0000000001
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:174)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:171)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:171)
   [junit4]   2> 	at org.apache.solr.cloud.ElectionContext.cancelElection(ElectionContext.java:68)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:294)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:164)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:108)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 340808 T146 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 340808 T146 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> NOTE: test params are: codec=Appending, sim=DefaultSimilarity, locale=in_ID, timezone=America/Guatemala
   [junit4]   2> NOTE: Linux 3.8.0-37-generic i386/Oracle Corporation 1.7.0_51 (32-bit)/cpus=8,threads=1,free=13111832,total=46637056
   [junit4]   2> NOTE: All tests run in this JVM: [DirectUpdateHandlerTest, FileUtilsTest, IndexSchemaRuntimeFieldTest, TestOmitPositions, TestWordDelimiterFilterFactory, JSONWriterTest, TestNumberUtils, ShardSplitTest]
   [junit4] Completed on J1 in 326.45s, 1 test, 1 error <<< FAILURES!

[...truncated 1005 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/build.xml:459: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/build.xml:439: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/solr/common-build.xml:505: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/lucene/common-build.xml:1278: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.7-Linux/lucene/common-build.xml:911: There were test failures: 365 suites, 1570 tests, 1 error, 35 ignored (7 assumptions)

Total time: 60 minutes 53 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.7.0_51 -client -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure