You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2013/06/04 17:22:48 UTC

[JENKINS] Lucene-Solr-trunk-Windows (32bit/jdk1.7.0_21) - Build # 2890 - Failure!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/2890/
Java: 32bit/jdk1.7.0_21 -server -XX:+UseG1GC

2 tests failed.
REGRESSION:  org.apache.solr.cloud.ChaosMonkeyShardSplitTest.testDistribSearch

Error Message:
Server at http://127.0.0.1:59855/ho_z/me returned non ok status:500, message:Server Error

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:59855/ho_z/me returned non ok status:500, message:Server Error
	at __randomizedtesting.SeedInfo.seed([E935AD6F4427DEE1:68D323773378BEDD]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
	at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
	at org.apache.solr.cloud.ChaosMonkeyShardSplitTest.doTest(ChaosMonkeyShardSplitTest.java:136)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)


REGRESSION:  org.apache.solr.spelling.suggest.SuggesterTSTTest.testSuggestions

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([E935AD6F4427DEE1:4FE9949A8A13891F]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:527)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:494)
	at org.apache.solr.spelling.suggest.SuggesterTest.testSuggestions(SuggesterTest.java:71)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//lst[@name='spellcheck']/lst[@name='suggestions']/lst[@name='ac']/int[@name='numFound'][.='2']
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">1</int></lst><lst name="spellcheck"><lst name="suggestions"/></lst>
</response>

	request was:q=ac&spellcheck.count=2&qt=/suggest_tst&spellcheck.onlyMorePopular=true
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:520)
	... 42 more




Build Log:
[...truncated 9221 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ChaosMonkeyShardSplitTest
[junit4:junit4]   2> 254235 T588 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /ho_z/me
[junit4:junit4]   2> 254242 T588 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\solrtest-ChaosMonkeyShardSplitTest-1370357044640
[junit4:junit4]   2> 254244 T588 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 254245 T589 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 254346 T588 oasc.ZkTestServer.run start zk server on port:59836
[junit4:junit4]   2> 254348 T588 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 254382 T595 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@826668 name:ZooKeeperConnection Watcher:127.0.0.1:59836 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 254383 T588 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 254384 T588 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 254406 T588 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 254413 T597 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17e25dd name:ZooKeeperConnection Watcher:127.0.0.1:59836/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 254413 T588 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 254413 T588 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 254424 T588 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 254432 T588 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 254439 T588 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 254449 T588 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 254452 T588 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 254480 T588 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\schema15.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 254482 T588 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 254495 T588 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 254496 T588 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 254511 T588 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 254512 T588 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 254527 T588 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 254528 T588 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 254539 T588 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 254539 T588 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 254550 T588 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 254551 T588 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 254560 T588 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 254561 T588 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 254569 T588 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 254571 T588 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 254601 T590 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13f0fa515a90001, likely client has closed socket
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 
[junit4:junit4]   2> 255016 T588 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 255028 T588 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:59843
[junit4:junit4]   2> 255029 T588 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 255029 T588 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 255030 T588 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1370357044984
[junit4:junit4]   2> 255030 T588 oasc.CoreContainer$Initializer.initialize looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1370357044984\solr.xml
[junit4:junit4]   2> 255031 T588 oasc.CoreContainer.<init> New CoreContainer 2257406
[junit4:junit4]   2> 255031 T588 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1370357044984\'
[junit4:junit4]   2> 255031 T588 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1370357044984\'
[junit4:junit4]   2> 255154 T588 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 255154 T588 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 255155 T588 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 255155 T588 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 255155 T588 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 255155 T588 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 255156 T588 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 255156 T588 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 255156 T588 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 255156 T588 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 255170 T588 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 255171 T588 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:59836/solr
[junit4:junit4]   2> 255172 T588 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 255175 T588 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 255187 T608 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5f3cdb name:ZooKeeperConnection Watcher:127.0.0.1:59836 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 255188 T588 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 255196 T588 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 255206 T588 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 255212 T610 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c21ecf name:ZooKeeperConnection Watcher:127.0.0.1:59836/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 255212 T588 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 255218 T588 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4]   2> 255227 T588 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4]   2> 255234 T588 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 255238 T588 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59843_ho_z%2Fme
[junit4:junit4]   2> 255240 T588 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59843_ho_z%2Fme
[junit4:junit4]   2> 255246 T588 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 255260 T588 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 255267 T588 oasc.Overseer.start Overseer (id=89807719278116867-127.0.0.1:59843_ho_z%2Fme-n_0000000000) starting
[junit4:junit4]   2> 255277 T588 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4]   2> 255288 T612 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 255290 T588 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 255297 T588 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4]   2> 255303 T588 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 255308 T611 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 255314 T613 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1370357044984\collection1
[junit4:junit4]   2> 255314 T613 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 255315 T613 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 255316 T613 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 255318 T613 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1370357044984\collection1\'
[junit4:junit4]   2> 255320 T613 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1370357044984/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 255321 T613 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1370357044984/collection1/lib/README' to classloader
[junit4:junit4]   2> 255379 T613 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 255450 T613 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 255453 T613 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 255460 T613 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 256113 T613 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 256117 T613 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 256120 T613 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 256146 T613 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 256196 T613 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 256197 T613 oasc.SolrCore.<init> [collection1] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1370357044984\collection1\, dataDir=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638/control/data\
[junit4:junit4]   2> 256197 T613 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d306ef
[junit4:junit4]   2> 256197 T613 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 256198 T613 oasc.CachingDirectoryFactory.get return new directory for .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638/control/data\
[junit4:junit4]   2> 256198 T613 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638/control/data\index/
[junit4:junit4]   2> 256198 T613 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638\control\data\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 256198 T613 oasc.CachingDirectoryFactory.get return new directory for .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638/control/data\index
[junit4:junit4]   2> 256203 T613 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1f855c5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1825c38),segFN=segments_1,generation=1}
[junit4:junit4]   2> 256204 T613 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 256207 T613 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 256207 T613 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 256208 T613 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 256209 T613 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 256209 T613 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 256209 T613 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 256209 T613 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 256211 T613 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 256211 T613 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 256222 T613 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 256229 T613 oass.SolrIndexSearcher.<init> Opening Searcher@265e80 main
[junit4:junit4]   2> 256230 T613 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638\control\data\tlog
[junit4:junit4]   2> 256231 T613 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 256231 T613 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 256236 T614 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@265e80 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 256237 T613 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 256237 T613 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 256816 T611 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 256818 T611 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"1",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:59843_ho_z%2Fme",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:59843/ho_z/me"}
[junit4:junit4]   2> 256818 T611 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 256818 T611 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4]   2> 256829 T610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 257242 T613 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 257242 T613 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:59843/ho_z/me collection:control_collection shard:shard1
[junit4:junit4]   2> 257243 T613 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 257266 T613 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 257273 T613 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 257273 T613 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 257273 T613 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:59843/ho_z/me/collection1/
[junit4:junit4]   2> 257274 T613 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 257274 T613 oasc.SyncStrategy.syncToMe http://127.0.0.1:59843/ho_z/me/collection1/ has no replicas
[junit4:junit4]   2> 257274 T613 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:59843/ho_z/me/collection1/
[junit4:junit4]   2> 257274 T613 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 258338 T611 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 258361 T610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 258406 T613 oasc.ZkController.register We are http://127.0.0.1:59843/ho_z/me/collection1/ and leader is http://127.0.0.1:59843/ho_z/me/collection1/
[junit4:junit4]   2> 258406 T613 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:59843/ho_z/me
[junit4:junit4]   2> 258406 T613 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 258406 T613 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 258406 T613 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 258410 T613 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 258413 T588 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
[junit4:junit4]   2> 258414 T588 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 258416 T588 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 258427 T588 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 258429 T588 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 258435 T617 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@814915 name:ZooKeeperConnection Watcher:127.0.0.1:59836/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 258435 T588 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 258438 T588 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 258443 T588 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 258865 T588 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 258871 T588 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:59855
[junit4:junit4]   2> 258871 T588 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 258872 T588 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 258872 T588 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1370357048839
[junit4:junit4]   2> 258872 T588 oasc.CoreContainer$Initializer.initialize looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1370357048839\solr.xml
[junit4:junit4]   2> 258873 T588 oasc.CoreContainer.<init> New CoreContainer 5245242
[junit4:junit4]   2> 258873 T588 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1370357048839\'
[junit4:junit4]   2> 258874 T588 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1370357048839\'
[junit4:junit4]   2> 258997 T588 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 258998 T588 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 258998 T588 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 258999 T588 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 258999 T588 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 258999 T588 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 259000 T588 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 259000 T588 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 259001 T588 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 259001 T588 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 259010 T588 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 259011 T588 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:59836/solr
[junit4:junit4]   2> 259011 T588 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 259015 T588 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 259031 T628 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@18e3248 name:ZooKeeperConnection Watcher:127.0.0.1:59836 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 259031 T588 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 259036 T588 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 259045 T588 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 259049 T630 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@143b527 name:ZooKeeperConnection Watcher:127.0.0.1:59836/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 259049 T588 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 259058 T588 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 259875 T611 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 259880 T611 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"1",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:59843_ho_z%2Fme",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:59843/ho_z/me"}
[junit4:junit4]   2> 259901 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 259902 T630 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 259901 T610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 260064 T588 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59855_ho_z%2Fme
[junit4:junit4]   2> 260070 T588 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59855_ho_z%2Fme
[junit4:junit4]   2> 260079 T610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 260080 T630 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 260082 T617 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 260083 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 260086 T610 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 260088 T630 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 260096 T631 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1370357048839\collection1
[junit4:junit4]   2> 260096 T631 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 260098 T631 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 260098 T631 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 260101 T631 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1370357048839\collection1\'
[junit4:junit4]   2> 260103 T631 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1370357048839/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 260104 T631 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1370357048839/collection1/lib/README' to classloader
[junit4:junit4]   2> 260161 T631 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 260272 T631 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 260275 T631 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 260281 T631 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 261088 T631 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 261090 T631 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 261093 T631 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 261103 T631 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 261156 T631 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 261157 T631 oasc.SolrCore.<init> [collection1] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1370357048839\collection1\, dataDir=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638/jetty1\
[junit4:junit4]   2> 261157 T631 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d306ef
[junit4:junit4]   2> 261157 T631 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 261158 T631 oasc.CachingDirectoryFactory.get return new directory for .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638/jetty1\
[junit4:junit4]   2> 261159 T631 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638/jetty1\index/
[junit4:junit4]   2> 261159 T631 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638\jetty1\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 261160 T631 oasc.CachingDirectoryFactory.get return new directory for .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638/jetty1\index
[junit4:junit4]   2> 261164 T631 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@19a6c1f lockFactory=org.apache.lucene.store.NativeFSLockFactory@1836c9d),segFN=segments_1,generation=1}
[junit4:junit4]   2> 261165 T631 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 261169 T631 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 261170 T631 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 261170 T631 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 261171 T631 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 261171 T631 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 261172 T631 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 261172 T631 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 261173 T631 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 261173 T631 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 261185 T631 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 261195 T631 oass.SolrIndexSearcher.<init> Opening Searcher@1c278 main
[junit4:junit4]   2> 261196 T631 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638\jetty1\tlog
[junit4:junit4]   2> 261198 T631 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 261198 T631 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 261201 T632 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1c278 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 261204 T631 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 261204 T631 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 261413 T611 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 261415 T611 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"2",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:59855_ho_z%2Fme",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:59855/ho_z/me"}
[junit4:junit4]   2> 261415 T611 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4]   2> 261415 T611 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4]   2> 261426 T610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 261426 T630 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 261426 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 262211 T631 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 262211 T631 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:59855/ho_z/me collection:collection1 shard:shard1
[junit4:junit4]   2> 262214 T631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 262237 T631 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 262243 T631 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 262243 T631 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 262244 T631 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:59855/ho_z/me/collection1/
[junit4:junit4]   2> 262244 T631 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 262244 T631 oasc.SyncStrategy.syncToMe http://127.0.0.1:59855/ho_z/me/collection1/ has no replicas
[junit4:junit4]   2> 262244 T631 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:59855/ho_z/me/collection1/
[junit4:junit4]   2> 262245 T631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 262937 T611 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 262975 T610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 262976 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 262977 T630 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 263029 T631 oasc.ZkController.register We are http://127.0.0.1:59855/ho_z/me/collection1/ and leader is http://127.0.0.1:59855/ho_z/me/collection1/
[junit4:junit4]   2> 263029 T631 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:59855/ho_z/me
[junit4:junit4]   2> 263029 T631 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 263029 T631 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 263030 T631 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 263032 T631 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 263034 T588 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
[junit4:junit4]   2> 263034 T588 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 263035 T588 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 263469 T588 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 263475 T588 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:59864
[junit4:junit4]   2> 263476 T588 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 263476 T588 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 263476 T588 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1370357053440
[junit4:junit4]   2> 263477 T588 oasc.CoreContainer$Initializer.initialize looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1370357053440\solr.xml
[junit4:junit4]   2> 263477 T588 oasc.CoreContainer.<init> New CoreContainer 16125335
[junit4:junit4]   2> 263478 T588 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1370357053440\'
[junit4:junit4]   2> 263478 T588 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1370357053440\'
[junit4:junit4]   2> 263591 T588 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 263591 T588 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 263591 T588 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 263591 T588 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 263592 T588 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 263593 T588 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 263593 T588 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 263594 T588 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 263594 T588 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 263594 T588 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 263604 T588 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 263604 T588 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:59836/solr
[junit4:junit4]   2> 263605 T588 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 263607 T588 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 263626 T644 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d1057 name:ZooKeeperConnection Watcher:127.0.0.1:59836 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 263627 T588 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 263631 T588 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 263632 T590 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13f0fa515a90007, likely client has closed socket
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 
[junit4:junit4]   2> 263641 T588 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 263645 T646 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a566ab name:ZooKeeperConnection Watcher:127.0.0.1:59836/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 263645 T588 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 263655 T588 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 264494 T611 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 264496 T611 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"2",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:59855_ho_z%2Fme",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:59855/ho_z/me"}
[junit4:junit4]   2> 264510 T610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 264511 T646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 264511 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 264513 T630 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 264660 T588 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59864_ho_z%2Fme
[junit4:junit4]   2> 264684 T588 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59864_ho_z%2Fme
[junit4:junit4]   2> 264696 T610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 264697 T646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 264697 T630 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 264701 T617 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 264702 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 264706 T610 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 264707 T646 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 264708 T630 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 264722 T647 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1370357053440\collection1
[junit4:junit4]   2> 264722 T647 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 264725 T647 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 264725 T647 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 264731 T647 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1370357053440\collection1\'
[junit4:junit4]   2> 264731 T647 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1370357053440/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 264734 T647 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1370357053440/collection1/lib/README' to classloader
[junit4:junit4]   2> 264819 T647 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 264907 T647 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 264915 T647 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 264925 T647 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 265767 T647 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 265771 T647 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 265776 T647 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 265782 T647 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 265846 T647 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 265846 T647 oasc.SolrCore.<init> [collection1] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1370357053440\collection1\, dataDir=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638/jetty2\
[junit4:junit4]   2> 265847 T647 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d306ef
[junit4:junit4]   2> 265847 T647 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 265848 T647 oasc.CachingDirectoryFactory.get return new directory for .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638/jetty2\
[junit4:junit4]   2> 265848 T647 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638/jetty2\index/
[junit4:junit4]   2> 265848 T647 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638\jetty2\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 265849 T647 oasc.CachingDirectoryFactory.get return new directory for .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638/jetty2\index
[junit4:junit4]   2> 265855 T647 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@e7736 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1bc87a7),segFN=segments_1,generation=1}
[junit4:junit4]   2> 265856 T647 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 265860 T647 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 265860 T647 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 265861 T647 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 265862 T647 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 265863 T647 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 265863 T647 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 265863 T647 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 265864 T647 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 265864 T647 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 265877 T647 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 265885 T647 oass.SolrIndexSearcher.<init> Opening Searcher@1812b4f main
[junit4:junit4]   2> 265885 T647 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638\jetty2\tlog
[junit4:junit4]   2> 265887 T647 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 265887 T647 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 265894 T648 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1812b4f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 265896 T647 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 265897 T647 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 266028 T611 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 266029 T611 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"3",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:59864_ho_z%2Fme",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:59864/ho_z/me"}
[junit4:junit4]   2> 266030 T611 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 266030 T611 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4]   2> 266043 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 266043 T610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 266044 T646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 266043 T630 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 266902 T647 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 266902 T647 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:59864/ho_z/me collection:collection1 shard:shard2
[junit4:junit4]   2> 266904 T647 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 266921 T647 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 266929 T647 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 266930 T647 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 266930 T647 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:59864/ho_z/me/collection1/
[junit4:junit4]   2> 266930 T647 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 266930 T647 oasc.SyncStrategy.syncToMe http://127.0.0.1:59864/ho_z/me/collection1/ has no replicas
[junit4:junit4]   2> 266931 T647 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:59864/ho_z/me/collection1/
[junit4:junit4]   2> 266931 T647 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 267561 T611 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 267583 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 267584 T646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 267584 T610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 267583 T630 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 267609 T647 oasc.ZkController.register We are http://127.0.0.1:59864/ho_z/me/collection1/ and leader is http://127.0.0.1:59864/ho_z/me/collection1/
[junit4:junit4]   2> 267609 T647 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:59864/ho_z/me
[junit4:junit4]   2> 267609 T647 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 267610 T647 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 267610 T647 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 267614 T647 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 267615 T588 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
[junit4:junit4]   2> 267615 T588 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 267615 T588 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 268041 T588 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 268046 T588 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:59873
[junit4:junit4]   2> 268048 T588 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 268048 T588 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 268049 T588 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1370357058021
[junit4:junit4]   2> 268049 T588 oasc.CoreContainer$Initializer.initialize looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1370357058021\solr.xml
[junit4:junit4]   2> 268049 T588 oasc.CoreContainer.<init> New CoreContainer 5080592
[junit4:junit4]   2> 268049 T588 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1370357058021\'
[junit4:junit4]   2> 268051 T588 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1370357058021\'
[junit4:junit4]   2> 268165 T588 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 268166 T588 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 268166 T588 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 268166 T588 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 268166 T588 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 268167 T588 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 268168 T588 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 268168 T588 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 268169 T588 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 268169 T588 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 268180 T588 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 268180 T588 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:59836/solr
[junit4:junit4]   2> 268180 T588 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 268183 T588 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 268198 T660 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b013bc name:ZooKeeperConnection Watcher:127.0.0.1:59836 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 268199 T588 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 268204 T588 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 268214 T588 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 268218 T662 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1bc5d66 name:ZooKeeperConnection Watcher:127.0.0.1:59836/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 268218 T588 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 268228 T588 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 269143 T611 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 269144 T611 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"3",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:59864_ho_z%2Fme",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:59864/ho_z/me"}
[junit4:junit4]   2> 269193 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 269195 T630 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 269196 T610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 269195 T662 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 269196 T646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 269292 T588 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59873_ho_z%2Fme
[junit4:junit4]   2> 269299 T588 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59873_ho_z%2Fme
[junit4:junit4]   2> 269305 T662 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 269305 T646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 269306 T617 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 269306 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 269305 T610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 269305 T630 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 269315 T662 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 269316 T646 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 269316 T610 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 269319 T630 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 269329 T663 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1370357058021\collection1
[junit4:junit4]   2> 269329 T663 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 269330 T663 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 269331 T663 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 269334 T663 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1370357058021\collection1\'
[junit4:junit4]   2> 269336 T663 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1370357058021/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 269337 T663 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1370357058021/collection1/lib/README' to classloader
[junit4:junit4]   2> 269405 T663 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 269478 T663 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 269481 T663 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 269488 T663 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 270214 T663 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 270218 T663 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 270220 T663 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 270227 T663 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 270278 T663 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 270280 T663 oasc.SolrCore.<init> [collection1] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1370357058021\collection1\, dataDir=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638/jetty3\
[junit4:junit4]   2> 270280 T663 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d306ef
[junit4:junit4]   2> 270281 T663 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 270281 T663 oasc.CachingDirectoryFactory.get return new directory for .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638/jetty3\
[junit4:junit4]   2> 270281 T663 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638/jetty3\index/
[junit4:junit4]   2> 270282 T663 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638\jetty3\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 270283 T663 oasc.CachingDirectoryFactory.get return new directory for .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638/jetty3\index
[junit4:junit4]   2> 270287 T663 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@17a2b5c lockFactory=org.apache.lucene.store.NativeFSLockFactory@823d7e),segFN=segments_1,generation=1}
[junit4:junit4]   2> 270287 T663 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 270291 T663 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 270291 T663 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 270292 T663 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 270293 T663 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 270293 T663 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 270295 T663 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 270295 T663 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 270296 T663 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 270296 T663 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 270311 T663 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 270319 T663 oass.SolrIndexSearcher.<init> Opening Searcher@cb71f5 main
[junit4:junit4]   2> 270319 T663 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638\jetty3\tlog
[junit4:junit4]   2> 270321 T663 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 270321 T663 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 270327 T664 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@cb71f5 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 270329 T663 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 270329 T663 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 270860 T611 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 270863 T611 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"4",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:59873_ho_z%2Fme",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:59873/ho_z/me"}
[junit4:junit4]   2> 270863 T611 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 270864 T611 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4]   2> 270888 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 270889 T630 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 270889 T662 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 270890 T610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 270890 T646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 271434 T663 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 271434 T663 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:59873/ho_z/me collection:collection1 shard:shard1
[junit4:junit4]   2> 271451 T663 oasc.ZkController.register We are http://127.0.0.1:59873/ho_z/me/collection1/ and leader is http://127.0.0.1:59855/ho_z/me/collection1/
[junit4:junit4]   2> 271453 T663 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:59873/ho_z/me
[junit4:junit4]   2> 271453 T663 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 271455 T663 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C1909 name=collection1 org.apache.solr.core.SolrCore@1204f73 url=http://127.0.0.1:59873/ho_z/me/collection1 node=127.0.0.1:59873_ho_z%2Fme C1909_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:59873_ho_z%2Fme, base_url=http://127.0.0.1:59873/ho_z/me}
[junit4:junit4]   2> 271456 T665 C1909 P59873 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 271459 T665 C1909 P59873 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 271459 T665 C1909 P59873 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 271460 T665 C1909 P59873 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 271460 T663 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 271467 T665 C1909 P59873 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 271469 T588 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
[junit4:junit4]   2> 271469 T588 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 271470 T588 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 271480 T624 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4]   2> 271929 T588 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 271935 T588 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:59883
[junit4:junit4]   2> 271938 T588 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 271939 T588 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 271939 T588 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1370357061875
[junit4:junit4]   2> 271939 T588 oasc.CoreContainer$Initializer.initialize looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1370357061875\solr.xml
[junit4:junit4]   2> 271940 T588 oasc.CoreContainer.<init> New CoreContainer 31321304
[junit4:junit4]   2> 271940 T588 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1370357061875\'
[junit4:junit4]   2> 271940 T588 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1370357061875\'
[junit4:junit4]   2> 272180 T588 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 272180 T588 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 272181 T588 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 272181 T588 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 272181 T588 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 272182 T588 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 272182 T588 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 272182 T588 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 272183 T588 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 272183 T588 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 272193 T588 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 272193 T588 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:59836/solr
[junit4:junit4]   2> 272194 T588 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 272195 T588 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 272214 T677 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@950fc name:ZooKeeperConnection Watcher:127.0.0.1:59836 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 272215 T588 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 272222 T588 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 272222 T590 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13f0fa515a9000b, likely client has closed socket
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 
[junit4:junit4]   2> 272229 T588 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 272235 T679 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f170f name:ZooKeeperConnection Watcher:127.0.0.1:59836/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 272236 T588 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 272246 T588 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 272579 T611 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 272580 T611 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"4",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:59873_ho_z%2Fme",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:59873/ho_z/me"}
[junit4:junit4]   2> 272604 T679 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 272604 T662 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 272604 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 272604 T630 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 272606 T610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 272606 T646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 273354 T588 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59883_ho_z%2Fme
[junit4:junit4]   2> 273357 T588 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59883_ho_z%2Fme
[junit4:junit4]   2> 273362 T630 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 273364 T679 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 273364 T646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 273364 T610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 273365 T617 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 273366 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 273364 T662 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 273364 T679 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 273373 T630 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 273375 T646 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 273375 T662 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 273375 T610 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 273386 T680 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1370357061875\collection1
[junit4:junit4]   2> 273386 T680 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 273387 T680 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 273387 T680 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 273390 T680 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1370357061875\collection1\'
[junit4:junit4]   2> 273393 T680 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1370357061875/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 273394 T680 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1370357061875/collection1/lib/README' to classloader
[junit4:junit4]   2> 273457 T680 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 273530 T680 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 273533 T680 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 273540 T680 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 273682 T624 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4]   2> 273682 T624 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=4&state=recovering&nodeName=127.0.0.1:59873_ho_z%252Fme&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2202 
[junit4:junit4]   2> 274282 T680 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 274286 T680 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 274288 T680 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 274297 T680 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 274346 T680 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 274348 T680 oasc.SolrCore.<init> [collection1] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1370357061875\collection1\, dataDir=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638/jetty4\
[junit4:junit4]   2> 274348 T680 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d306ef
[junit4:junit4]   2> 274349 T680 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 274349 T680 oasc.CachingDirectoryFactory.get return new directory for .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638/jetty4\
[junit4:junit4]   2> 274350 T680 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638/jetty4\index/
[junit4:junit4]   2> 274350 T680 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638\jetty4\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 274352 T680 oasc.CachingDirectoryFactory.get return new directory for .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638/jetty4\index
[junit4:junit4]   2> 274356 T680 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1a28f15 lockFactory=org.apache.lucene.store.NativeFSLockFactory@eeaad8),segFN=segments_1,generation=1}
[junit4:junit4]   2> 274356 T680 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 274361 T680 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 274363 T680 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 274363 T680 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 274364 T680 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 274365 T680 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 274365 T680 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 274365 T680 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 274366 T680 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 274366 T680 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 274379 T680 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 274388 T680 oass.SolrIndexSearcher.<init> Opening Searcher@92eb4f main
[junit4:junit4]   2> 274388 T680 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638\jetty4\tlog
[junit4:junit4]   2> 274390 T680 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 274390 T680 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 274397 T681 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@92eb4f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 274399 T680 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 274399 T680 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> ASYNC  NEW_CORE C1910 name=collection1 org.apache.solr.core.SolrCore@1204f73 url=http://127.0.0.1:59873/ho_z/me/collection1 node=127.0.0.1:59873_ho_z%2Fme C1910_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:59873_ho_z%2Fme, base_url=http://127.0.0.1:59873/ho_z/me}
[junit4:junit4]   2> 275883 T665 C1910 P59873 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:59855/ho_z/me/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 275883 T665 C1910 P59873 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:59873/ho_z/me START replicas=[http://127.0.0.1:59855/ho_z/me/collection1/] nUpdates=100
[junit4:junit4]   2> 275885 T665 C1910 P59873 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 275885 T665 C1910 P59873 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 275885 T665 C1910 P59873 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 275885 T665 C1910 P59873 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 275885 T665 C1910 P59873 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 275886 T665 C1910 P59873 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:59855/ho_z/me/collection1/. core=collection1
[junit4:junit4]   2> 275886 T665 C1910 P59873 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C1911 name=collection1 org.apache.solr.core.SolrCore@144fae1 url=http://127.0.0.1:59855/ho_z/me/collection1 node=127.0.0.1:59855_ho_z%2Fme C1911_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:59855_ho_z%2Fme, base_url=http://127.0.0.1:59855/ho_z/me, leader=true}
[junit4:junit4]   2> 275894 T624 C1911 P59855 oasc.SolrCore.execute [collection1] webapp=/ho_z/me path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 275898 T626 C1911 P59855 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 275902 T626 C1911 P59855 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@19a6c1f lockFactory=org.apache.lucene.store.NativeFSLockFactory@1836c9d),segFN=segments_1,generation=1}
[junit4:junit4]   2> 275904 T626 C1911 P59855 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 275905 T626 C1911 P59855 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@19a6c1f lockFactory=org.apache.lucene.store.NativeFSLockFactory@1836c9d),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@19a6c1f lockFactory=org.apache.lucene.store.NativeFSLockFactory@1836c9d),segFN=segments_2,generation=2}
[junit4:junit4]   2> 275905 T626 C1911 P59855 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 275905 T626 C1911 P59855 oass.SolrIndexSearcher.<init> Opening Searcher@12488bd realtime
[junit4:junit4]   2> 275906 T626 C1911 P59855 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 275906 T626 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 8
[junit4:junit4]   2> 275907 T665 C1910 P59873 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 275907 T665 C1910 P59873 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 275912 T623 C1911 P59855 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 275912 T623 C1911 P59855 oasc.SolrCore.execute [collection1] webapp=/ho_z/me path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 275913 T665 C1910 P59873 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 275913 T665 C1910 P59873 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 275913 T665 C1910 P59873 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 275915 T622 C1911 P59855 oasc.SolrCore.execute [collection1] webapp=/ho_z/me path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0 
[junit4:junit4]   2> 275916 T665 C1910 P59873 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 275917 T665 C1910 P59873 oasc.CachingDirectoryFactory.get return new directory for .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638/jetty3\index.20130604194426312
[junit4:junit4]   2> 275917 T665 C1910 P59873 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@192d3b1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@cf0f12) fullCopy=false
[junit4:junit4]   2> 275920 T625 C1911 P59855 oasc.SolrCore.execute [collection1] webapp=/ho_z/me path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 275922 T665 C1910 P59873 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 275927 T665 C1910 P59873 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 275927 T665 C1910 P59873 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 275932 T611 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 275934 T611 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"5",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:59883_ho_z%2Fme",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:59883/ho_z/me"}
[junit4:junit4]   2> 275934 T611 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 275934 T611 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4]   2> 275934 T665 C1910 P59873 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@17a2b5c lockFactory=org.apache.lucene.store.NativeFSLockFactory@823d7e),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@17a2b5c lockFactory=org.apache.lucene.store.NativeFSLockFactory@823d7e),segFN=segments_2,generation=2}
[junit4:junit4]   2> 275935 T665 C1910 P59873 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 275936 T665 C1910 P59873 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 275936 T665 C1910 P59873 oass.SolrIndexSearcher.<init> Opening Searcher@14cfe38 main
[junit4:junit4]   2> 275937 T664 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@14cfe38 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 275938 T665 C1910 P59873 oasc.CachingDirectoryFactory.closeCacheValue looking to close .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638/jetty3\index.20130604194426312 [CachedDir<<refCount=0;path=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638/jetty3\index.20130604194426312;done=true>>]
[junit4:junit4]   2> 275938 T665 C1910 P59873 oasc.CachingDirectoryFactory.close Closing directory: .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638/jetty3\index.20130604194426312
[junit4:junit4]   2> 275938 T665 C1910 P59873 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638/jetty3\index.20130604194426312
[junit4:junit4]   2> 275938 T665 C1910 P59873 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 275938 T665 C1910 P59873 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 275939 T665 C1910 P59873 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 275939 T665 C1910 P59873 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 275945 T665 C1910 P59873 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 275951 T611 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"4",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:59873_ho_z%2Fme",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:59873/ho_z/me"}
[junit4:junit4]   2> 275962 T630 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 275962 T662 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 275963 T610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 275963 T646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 275962 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 275962 T679 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 276603 T680 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 276603 T680 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:59883/ho_z/me collection:collection1 shard:shard2
[junit4:junit4]   2> 276617 T680 oasc.ZkController.register We are http://127.0.0.1:59883/ho_z/me/collection1/ and leader is http://127.0.0.1:59864/ho_z/me/collection1/
[junit4:junit4]   2> 276617 T680 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:59883/ho_z/me
[junit4:junit4]   2> 276618 T680 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 276619 T680 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C1912 name=collection1 org.apache.solr.core.SolrCore@5da57b url=http://127.0.0.1:59883/ho_z/me/collection1 node=127.0.0.1:59883_ho_z%2Fme C1912_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:59883_ho_z%2Fme, base_url=http://127.0.0.1:59883/ho_z/me}
[junit4:junit4]   2> 276622 T684 C1912 P59883 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 276623 T684 C1912 P59883 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 276623 T684 C1912 P59883 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 276624 T680 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 276624 T684 C1912 P59883 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 276628 T588 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
[junit4:junit4]   2> 276628 T588 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 276629 T588 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 276629 T684 C1912 P59883 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 276639 T588 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 276642 T588 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4]   2> 276642 T588 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4]   2> 276644 T588 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 276651 T641 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4]   2> 277636 T611 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 277639 T611 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"5",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:59883_ho_z%2Fme",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:59883/ho_z/me"}
[junit4:junit4]   2> 277667 T679 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 277667 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 277669 T630 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 277671 T662 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 277671 T610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 277672 T646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 277751 T641 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4]   2> 277751 T641 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=5&state=recovering&nodeName=127.0.0.1:59883_ho_z%252Fme&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1100 
[junit4:junit4]   2> 277754 T588 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 278862 T588 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C1912_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:59883_ho_z%2Fme, base_url=http://127.0.0.1:59883/ho_z/me}
[junit4:junit4]   2> 279923 T684 C1912 P59883 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:59864/ho_z/me/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 279923 T684 C1912 P59883 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:59883/ho_z/me START replicas=[http://127.0.0.1:59864/ho_z/me/collection1/] nUpdates=100
[junit4:junit4]   2> 279924 T684 C1912 P59883 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 279924 T684 C1912 P59883 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 279924 T684 C1912 P59883 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 279924 T684 C1912 P59883 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 279924 T684 C1912 P59883 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 279925 T684 C1912 P59883 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:59864/ho_z/me/collection1/. core=collection1
[junit4:junit4]   2> 279925 T684 C1912 P59883 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C1913 name=collection1 org.apache.solr.core.SolrCore@14ec436 url=http://127.0.0.1:59864/ho_z/me/collection1 node=127.0.0.1:59864_ho_z%2Fme C1913_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:59864_ho_z%2Fme, base_url=http://127.0.0.1:59864/ho_z/me, leader=true}
[junit4:junit4]   2> 279940 T641 C1913 P59864 oasc.SolrCore.execute [collection1] webapp=/ho_z/me path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 279943 T639 C1913 P59864 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 279945 T588 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 279949 T639 C1913 P59864 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@e7736 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1bc87a7),segFN=segments_1,generation=1}
[junit4:junit4]   2> 279951 T639 C1913 P59864 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 279955 T639 C1913 P59864 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@e7736 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1bc87a7),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@e7736 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1bc87a7),segFN=segments_2,generation=2}
[junit4:junit4]   2> 279955 T639 C1913 P59864 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 279955 T639 C1913 P59864 oass.SolrIndexSearcher.<init> Opening Searcher@a8106a realtime
[junit4:junit4]   2> 279956 T639 C1913 P59864 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 279956 T639 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 13
[junit4:junit4]   2> 279957 T684 C1912 P59883 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 279957 T684 C1912 P59883 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 279962 T640 C1913 P59864 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 279962 T640 C1913 P59864 oasc.SolrCore.execute [collection1] webapp=/ho_z/me path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 279963 T684 C1912 P59883 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 279963 T684 C1912 P59883 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 279963 T684 C1912 P59883 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 279966 T642 C1913 P59864 oasc.SolrCore.execute [collection1] webapp=/ho_z/me path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0 
[junit4:junit4]   2> 279967 T684 C1912 P59883 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 279968 T684 C1912 P59883 oasc.CachingDirectoryFactory.get return new directory for .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638/jetty4\index.20130604194430363
[junit4:junit4]   2> 279968 T684 C1912 P59883 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@c160d9 lockFactory=org.apache.lucene.store.NativeFSLockFactory@48fbe9) fullCopy=false
[junit4:junit4]   2> 279971 T636 C1913 P59864 oasc.SolrCore.execute [collection1] webapp=/ho_z/me path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 279972 T684 C1912 P59883 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 279974 T684 C1912 P59883 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 279974 T684 C1912 P59883 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 279977 T684 C1912 P59883 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1a28f15 lockFactory=org.apache.lucene.store.NativeFSLockFactory@eeaad8),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1a28f15 lockFactory=org.apache.lucene.store.NativeFSLockFactory@eeaad8),segFN=segments_2,generation=2}
[junit4:junit4]   2> 279978 T684 C1912 P59883 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 279978 T684 C1912 P59883 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 279978 T684 C1912 P59883 oass.SolrIndexSearcher.<init> Opening Searcher@95bdd0 main
[junit4:junit4]   2> 279979 T681 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@95bdd0 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 279981 T684 C1912 P59883 oasc.CachingDirectoryFactory.closeCacheValue looking to close .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638/jetty4\index.20130604194430363 [CachedDir<<refCount=0;path=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638/jetty4\index.20130604194430363;done=true>>]
[junit4:junit4]   2> 279981 T684 C1912 P59883 oasc.CachingDirectoryFactory.close Closing directory: .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638/jetty4\index.20130604194430363
[junit4:junit4]   2> 279981 T684 C1912 P59883 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370357044638/jetty4\index.20130604194430363
[junit4:junit4]   2> 279981 T684 C1912 P59883 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 279981 T684 C1912 P59883 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 279982 T684 C1912 P59883 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 279982 T684 C1912 P59883 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 279990 T684 C1912 P59883 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 280875 T611 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 280876 T611 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"5",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:59883_ho_z%2Fme",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:59883/ho_z/me"}
[junit4:junit4]   2> 280891 T679 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 280892 T630 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 280892 T610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 280892 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 280892 T662 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 280893 T646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 280945 T588 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 280946 T588 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4]   2> ASYNC  NEW_CORE C1914 name=collection1 org.apache.solr.core.SolrCore@1dbcfd2 url=http://127.0.0.1:59843/ho_z/me/collection1 node=127.0.0.1:59843_ho_z%2Fme C1914_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:59843_ho_z%2Fme, base_url=http://127.0.0.1:59843/ho_z/me, leader=true}
[junit4:junit4]   2> 280960 T605 C1914 P59843 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 280964 T605 C1914 P59843 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1f855c5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1825c38),segFN=segments_1,generation=1}
[junit4:junit4]   2> 280965 T605 C1914 P59843 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 280965 T605 C1914 P59843 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1f855c5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1825c38),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1f855c5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1825c38),segFN=segments_2,generation=2}
[junit4:junit4]   2> 280966 T605 C1914 P59843 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 280966 T605 C1914 P59843 oass.SolrIndexSearcher.<init> Opening Searcher@1b0d7c2 main
[junit4:junit4]   2> 280967 T605 C1914 P59843 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 280968 T614 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1b0d7c2 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 280969 T605 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4]   2> 280977 T639 C1913 P59864 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 280978 T639 C1913 P59864 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@e7736 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1bc87a7),segFN=segments_2,generation=2}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@e7736 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1bc87a7),segFN=segments_3,generation=3}
[junit4:junit4]   2> 280978 T639 C1913 P59864 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4]   2> 280978 T639 C1913 P59864 oass.SolrIndexSearcher.<init> Opening Searcher@c424b4 main
[junit4:junit4]   2> 280979 T639 C1913 P59864 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 280980 T648 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@c424b4 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 280981 T639 C1913 P59864 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:59855/ho_z/me/collection1/, StdNode: http://127.0.0.1:59873/ho_z/me/collection1/, StdNode: http://127.0.0.1:59883/ho_z/me/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4]   2> 280986 T622 C1911 P59855 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2>  C1910_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:59873_ho_z%2Fme, base_url=http://127.0.0.1:59873/ho_z/me}
[junit4:junit4]   2> 280993 T657 C1910 P59873 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2>  C1912_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:59883_ho_z%2Fme, base_url=http://127.0.0.1:59883/ho_z/me}
[junit4:junit4]   2> 280994 T671 C1912 P59883 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 280995 T622 C1911 P59855 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@19a6c1f lockFactory=org.apache.lucene.store.NativeFSLockFactory@1836c9d),segFN=segments_2,generation=2}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@19a6c1f lockFactory=org.apache.lucene.store.NativeFSLockFactory@1836c9d),segFN=segments_3,generation=3}
[junit4:junit4]   2> 280995 T622 C1911 P59855 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4]   2> 280997 T622 C1911 P59855 oass.SolrIndexSearcher.<init> Opening Searcher@a84820 main
[junit4:junit4]   2> 280997 T657 C1910 P59873 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@17a2b5c lockFactory=org.apache.lucene.store.NativeFSLockFactory@823d7e),segFN=segments_2,generation=2}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@17a2b5c lockFactory=org.apache.lucene.store.NativeFSLockFactory@823d7e),segFN=segments_3,generation=3}
[junit4:junit4]   2> 280998 T657 C1910 P59873 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4]   2> 280997 T622 C1911 P59855 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 280997 T671 C1912 P59883 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1a28f15 lockFactory=org.apache.lucene.store.NativeFSLockFactory@eeaad8),segFN=segments_2,generation=2}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1a28f15 lockFactory=org.apache.lucene.store.NativeFSLockFactory@eeaad8),segFN=segments_3,generation=3}
[junit4:junit4]   2> 280999 T671 C1912 P59883 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4]   2> 280999 T657 C1910 P59873 oass.SolrIndexSearcher.<init> Opening Searcher@18b09bf main
[junit4:junit4]   2> 281001 T671 C1912 P59883 oass.SolrIndexSearcher.<init> Opening Searcher@10bfcf main
[junit4:junit4]   2> 281001 T671 C1912 P59883 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 280998 T632 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@a84820 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 281002 T664 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@18b09bf main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 281003 T681 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@10bfcf main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 281003 T622 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 17
[junit4:junit4]   2> 281003 T671 C1912 P59883 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4]   2> 281001 T657 C1910 P59873 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 281004 T657 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 11
[junit4:junit4]   2> 281005 T639 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 28
[junit4:junit4]   2> 281006 T588 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 281010 T625 C1911 P59855 oasc.SolrCore.execute [collection1] webapp=/ho_z/me path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0 
[junit4:junit4]   2> 281013 T658 C1910 P59873 oasc.SolrCore.execute [collection1] webapp=/ho_z/me path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 281015 T640 C1913 P59864 oasc.SolrCore.execute [collection1] webapp=/ho_z/me path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0 
[junit4:junit4]   2> 281016 T672 C1912 P59883 oasc.SolrCore.execute [collection1] webapp=/ho_z/me path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0 
[junit4:junit4]   2> 283009 T606 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1436923538597478400)} 0 9
[junit4:junit4]   2> 283020 T656 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={update.distrib=FROMLEADER&_version_=-1436923538604818432&update.from=http://127.0.0.1:59855/ho_z/me/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1436923538604818432)} 0 3
[junit4:junit4]   2> 283026 T670 C1912 P59883 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={update.distrib=FROMLEADER&_version_=-1436923538611109888&update.from=http://127.0.0.1:59864/ho_z/me/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1436923538611109888)} 0 2
[junit4:junit4]   2> 283026 T641 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1436923538611109888)} 0 9
[junit4:junit4]   2> 283027 T623 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1436923538604818432)} 0 17
[junit4:junit4]   2> 283031 T605 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[0 (1436923538623692800)]} 0 2
[junit4:junit4]   2> 283041 T657 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1436923538631032832)]} 0 2
[junit4:junit4]   2> 283042 T622 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[0 (1436923538631032832)]} 0 6
[junit4:junit4]   2> 283042 T639 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[0]} 0 9
[junit4:junit4]   2> 283046 T603 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[1 (1436923538640470016)]} 0 1
[junit4:junit4]   2> 283053 T658 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1436923538644664320)]} 0 1
[junit4:junit4]   2> 283053 T625 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[1 (1436923538644664320)]} 0 4
[junit4:junit4]   2> 283055 T601 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[2 (1436923538650955776)]} 0 0
[junit4:junit4]   2> 283061 T671 C1912 P59883 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1436923538654101504)]} 0 1
[junit4:junit4]   2> 283062 T640 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[2 (1436923538654101504)]} 0 4
[junit4:junit4]   2> 283066 T599 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[3 (1436923538662490112)]} 0 0
[junit4:junit4]   2> 283077 T672 C1912 P59883 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1436923538669830144)]} 0 1
[junit4:junit4]   2> 283078 T642 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[3 (1436923538669830144)]} 0 5
[junit4:junit4]   2> 283080 T624 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[3]} 0 10
[junit4:junit4]   2> 283083 T604 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[4 (1436923538680315904)]} 0 0
[junit4:junit4]   2> 283089 T653 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1436923538683461632)]} 0 0
[junit4:junit4]   2> 283090 T626 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[4 (1436923538683461632)]} 0 4
[junit4:junit4]   2> 283094 T606 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[5 (1436923538691850240)]} 0 0
[junit4:junit4]   2> 283103 T673 C1912 P59883 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[5 (1436923538698141696)]} 0 0
[junit4:junit4]   2> 283104 T636 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[5 (1436923538698141696)]} 0 4
[junit4:junit4]   2> 283105 T621 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[5]} 0 8
[junit4:junit4]   2> 283109 T605 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[6 (1436923538707578880)]} 0 0
[junit4:junit4]   2> 283121 T674 C1912 P59883 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[6 (1436923538712821760)]} 0 0
[junit4:junit4]   2> 283123 T638 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[6 (1436923538712821760)]} 0 9
[junit4:junit4]   2> 283124 T623 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[6]} 0 12
[junit4:junit4]   2> 283128 T603 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[7 (1436923538727501824)]} 0 0
[junit4:junit4]   2> 283135 T675 C1912 P59883 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[7 (1436923538731696128)]} 0 0
[junit4:junit4]   2> 283136 T641 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[7 (1436923538731696128)]} 0 4
[junit4:junit4]   2> 283139 T601 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[8 (1436923538739036160)]} 0 0
[junit4:junit4]   2> 283148 T654 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1436923538745327616)]} 0 0
[junit4:junit4]   2> 283149 T622 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[8 (1436923538745327616)]} 0 4
[junit4:junit4]   2> 283150 T639 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[8]} 0 8
[junit4:junit4]   2> 283155 T599 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[9 (1436923538755813376)]} 0 1
[junit4:junit4]   2> 283162 T670 C1912 P59883 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[9 (1436923538761056256)]} 0 0
[junit4:junit4]   2> 283163 T640 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[9 (1436923538761056256)]} 0 3
[junit4:junit4]   2> 283167 T604 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[10 (1436923538767347712)]} 0 1
[junit4:junit4]   2> 283173 T655 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1436923538772590592)]} 0 0
[junit4:junit4]   2> 283174 T625 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[10 (1436923538772590592)]} 0 3
[junit4:junit4]   2> 283175 T642 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[10]} 0 5
[junit4:junit4]   2> 283177 T606 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[11 (1436923538778882048)]} 0 0
[junit4:junit4]   2> 283182 T656 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[11 (1436923538782027776)]} 0 0
[junit4:junit4]   2> 283184 T624 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[11 (1436923538782027776)]} 0 4
[junit4:junit4]   2> 283186 T605 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[12 (1436923538788319232)]} 0 0
[junit4:junit4]   2> 283191 T657 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[12 (1436923538791464960)]} 0 0
[junit4:junit4]   2> 283192 T626 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[12 (1436923538791464960)]} 0 3
[junit4:junit4]   2> 283194 T603 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[13 (1436923538796707840)]} 0 0
[junit4:junit4]   2> 283200 T658 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[13 (1436923538799853568)]} 0 0
[junit4:junit4]   2> 283201 T621 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[13 (1436923538799853568)]} 0 4
[junit4:junit4]   2> 283203 T601 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[14 (1436923538806145024)]} 0 0
[junit4:junit4]   2> 283208 T653 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[14 (1436923538809290752)]} 0 0
[junit4:junit4]   2> 283209 T623 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[14 (1436923538809290752)]} 0 3
[junit4:junit4]   2> 283212 T599 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[15 (1436923538815582208)]} 0 0
[junit4:junit4]   2> 283217 T654 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[15 (1436923538818727936)]} 0 0
[junit4:junit4]   2> 283219 T622 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[15 (1436923538818727936)]} 0 4
[junit4:junit4]   2> 283221 T604 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[16 (1436923538825019392)]} 0 0
[junit4:junit4]   2> 283229 T655 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[16 (1436923538830262272)]} 0 0
[junit4:junit4]   2> 283230 T625 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[16 (1436923538830262272)]} 0 4
[junit4:junit4]   2> 283230 T636 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[16]} 0 6
[junit4:junit4]   2> 283232 T606 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[17 (1436923538836553728)]} 0 0
[junit4:junit4]   2> 283240 T671 C1912 P59883 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[17 (1436923538841796608)]} 0 0
[junit4:junit4]   2> 283241 T638 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[17 (1436923538841796608)]} 0 4
[junit4:junit4]   2> 283242 T624 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[17]} 0 7
[junit4:junit4]   2> 283244 T605 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[18 (1436923538849136640)]} 0 0
[junit4:junit4]   2> 283250 T672 C1912 P59883 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[18 (1436923538852282368)]} 0 0
[junit4:junit4]   2> 283252 T641 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[18 (1436923538852282368)]} 0 5
[junit4:junit4]   2> 283256 T603 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[19 (1436923538861719552)]} 0 0
[junit4:junit4]   2> 283262 T673 C1912 P59883 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[19 (1436923538864865280)]} 0 0
[junit4:junit4]   2> 283263 T639 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[19 (1436923538864865280)]} 0 4
[junit4:junit4]   2> 283266 T601 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[20 (1436923538872205312)]} 0 0
[junit4:junit4]   2> 283272 T656 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[20 (1436923538875351040)]} 0 0
[junit4:junit4]   2> 283273 T626 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[20 (1436923538875351040)]} 0 4
[junit4:junit4]   2> 283275 T599 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[21 (1436923538881642496)]} 0 0
[junit4:junit4]   2> 283282 T674 C1912 P59883 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[21 (1436923538886885376)]} 0 0
[junit4:junit4]   2> 283283 T640 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[21 (1436923538886885376)]} 0 3
[junit4:junit4]   2> 283284 T621 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[21]} 0 6
[junit4:junit4]   2> 283286 T604 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[22 (1436923538893176832)]} 0 0
[junit4:junit4]   2> 283291 T657 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[22 (1436923538895273984)]} 0 0
[junit4:junit4]   2> 283293 T623 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[22 (1436923538895273984)]} 0 5
[junit4:junit4]   2> 283295 T606 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[23 (1436923538902614016)]} 0 0
[junit4:junit4]   2> 283303 T675 C1912 P59883 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[23 (1436923538907856896)]} 0 0
[junit4:junit4]   2> 283304 T642 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[23 (1436923538907856896)]} 0 4
[junit4:junit4]   2> 283305 T622 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[23]} 0 7
[junit4:junit4]   2> 283307 T605 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[24 (1436923538915196928)]} 0 0
[junit4:junit4]   2> 283316 T658 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[24 (1436923538921488384)]} 0 0
[junit4:junit4]   2> 283317 T625 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[24 (1436923538921488384)]} 0 4
[junit4:junit4]   2> 283318 T636 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[24]} 0 8
[junit4:junit4]   2> 283320 T603 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[25 (1436923538928828416)]} 0 0
[junit4:junit4]   2> 283326 T670 C1912 P59883 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[25 (1436923538931974144)]} 0 0
[junit4:junit4]   2> 283327 T638 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[25 (1436923538931974144)]} 0 4
[junit4:junit4]   2> 283330 T601 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[26 (1436923538939314176)]} 0 0
[junit4:junit4]   2> 283336 T671 C1912 P59883 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[26 (1436923538942459904)]} 0 0
[junit4:junit4]   2> 283337 T641 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[26 (1436923538942459904)]} 0 4
[junit4:junit4]   2> 283339 T599 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[27 (1436923538948751360)]} 0 0
[junit4:junit4]   2> 283345 T653 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[27 (1436923538951897088)]} 0 0
[junit4:junit4]   2> 283346 T624 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[27 (1436923538951897088)]} 0 4
[junit4:junit4]   2> 283348 T604 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[28 (1436923538958188544)]} 0 0
[junit4:junit4]   2> 283354 T654 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[28 (1436923538961334272)]} 0 0
[junit4:junit4]   2> 283355 T626 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[28 (1436923538961334272)]} 0 4
[junit4:junit4]   2> 283357 T606 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[29 (1436923538967625728)]} 0 0
[junit4:junit4]   2> 283365 T672 C1912 P59883 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[29 (1436923538972868608)]} 0 0
[junit4:junit4]   2> 283366 T639 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[29 (1436923538972868608)]} 0 4
[junit4:junit4]   2> 283367 T621 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[29]} 0 7
[junit4:junit4]   2> 283375 T605 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[30 (1436923538980208640)]} 0 6
[junit4:junit4]   2> 283380 T673 C1912 P59883 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[30 (1436923538988597248)]} 0 0
[junit4:junit4]   2> 283381 T640 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[30 (1436923538988597248)]} 0 4
[junit4:junit4]   2> 283384 T603 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[31 (1436923538995937280)]} 0 0
[junit4:junit4]   2> 283393 T674 C1912 P59883 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[31 (1436923539002228736)]} 0 0
[junit4:junit4]   2> 283394 T642 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[31 (1436923539002228736)]} 0 4
[junit4:junit4]   2> 283395 T623 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[31]} 0 8
[junit4:junit4]   2> 283398 T601 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[32 (1436923539010617344)]} 0 0
[junit4:junit4]   2> 283404 T655 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[32 (1436923539013763072)]} 0 0
[junit4:junit4]   2> 283404 T625 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[32 (1436923539013763072)]} 0 3
[junit4:junit4]   2> 283406 T599 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[33 (1436923539019005952)]} 0 0
[junit4:junit4]   2> 283411 T675 C1912 P59883 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[33 (1436923539022151680)]} 0 0
[junit4:junit4]   2> 283412 T636 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[33 (1436923539022151680)]} 0 3
[junit4:junit4]   2> 283414 T606 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[34 (1436923539027394560)]} 0 0
[junit4:junit4]   2> 283419 T657 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[34 (1436923539030540288)]} 0 0
[junit4:junit4]   2> 283420 T624 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[34 (1436923539030540288)]} 0 3
[junit4:junit4]   2> 283424 T605 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[35 (1436923539037880320)]} 0 0
[junit4:junit4]   2> 283430 T658 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[35 (1436923539041026048)]} 0 0
[junit4:junit4]   2> 283431 T626 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[35 (1436923539041026048)]} 0 4
[junit4:junit4]   2> 283433 T603 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[36 (1436923539047317504)]} 0 0
[junit4:junit4]   2> 283438 T653 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[36 (1436923539050463232)]} 0 0
[junit4:junit4]   2> 283439 T621 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[36 (1436923539050463232)]} 0 3
[junit4:junit4]   2> 283441 T601 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[37 (1436923539055706112)]} 0 0
[junit4:junit4]   2> 283449 T671 C1912 P59883 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[37 (1436923539060948992)]} 0 0
[junit4:junit4]   2> 283450 T641 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[37 (1436923539060948992)]} 0 4
[junit4:junit4]   2> 283451 T623 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[37]} 0 7
[junit4:junit4]   2> 283453 T604 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[38 (1436923539068289024)]} 0 0
[junit4:junit4]   2> 283461 T654 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[38 (1436923539073531904)]} 0 0
[junit4:junit4]   2> 283462 T622 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[38 (1436923539073531904)]} 0 4
[junit4:junit4]   2> 283463 T639 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[38]} 0 7
[junit4:junit4]   2> 283465 T599 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[39 (1436923539080871936)]} 0 0
[junit4:junit4]   2> 283474 T672 C1912 P59883 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[39 (1436923539087163392)]} 0 0
[junit4:junit4]   2> 283476 T640 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[39 (1436923539087163392)]} 0 5
[junit4:junit4]   2> 283477 T625 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[39]} 0 9
[junit4:junit4]   2> 283480 T606 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[40 (1436923539096600576)]} 0 0
[junit4:junit4]   2> 283487 T656 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[40 (1436923539099746304)]} 0 0
[junit4:junit4]   2> 283488 T624 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[40 (1436923539099746304)]} 0 5
[junit4:junit4]   2> 283490 T605 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[41 (1436923539107086336)]} 0 0
[junit4:junit4]   2> 283495 T655 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[41 (1436923539109183488)]} 0 0
[junit4:junit4]   2> 283497 T626 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[41 (1436923539109183488)]} 0 5
[junit4:junit4]   2> 283499 T603 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[42 (1436923539116523520)]} 0 0
[junit4:junit4]   2> 283505 T657 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[42 (1436923539119669248)]} 0 0
[junit4:junit4]   2> 283506 T621 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[42 (1436923539119669248)]} 0 4
[junit4:junit4]   2> 283509 T601 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[43 (1436923539127009280)]} 0 0
[junit4:junit4]   2> 283515 T658 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[43 (1436923539130155008)]} 0 0
[junit4:junit4]   2> 283516 T623 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[43 (1436923539130155008)]} 0 4
[junit4:junit4]   2> 283518 T604 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[44 (1436923539136446464)]} 0 0
[junit4:junit4]   2> 283523 T653 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[44 (1436923539139592192)]} 0 0
[junit4:junit4]   2> 283525 T622 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[44 (1436923539139592192)]} 0 4
[junit4:junit4]   2> 283527 T599 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[45 (1436923539145883648)]} 0 0
[junit4:junit4]   2> 283536 T673 C1912 P59883 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[45 (1436923539152175104)]} 0 0
[junit4:junit4]   2> 283537 T642 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[45 (1436923539152175104)]} 0 4
[junit4:junit4]   2> 283538 T625 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[45]} 0 8
[junit4:junit4]   2> 283540 T606 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[46 (1436923539159515136)]} 0 0
[junit4:junit4]   2> 283549 T654 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[46 (1436923539165806592)]} 0 0
[junit4:junit4]   2> 283550 T624 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[46 (1436923539165806592)]} 0 4
[junit4:junit4]   2> 283550 T638 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[46]} 0 6
[junit4:junit4]   2> 283552 T605 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[47 (1436923539172098048)]} 0 0
[junit4:junit4]   2> 283558 T656 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[47 (1436923539175243776)]} 0 0
[junit4:junit4]   2> 283559 T626 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[47 (1436923539175243776)]} 0 4
[junit4:junit4]   2> 283561 T603 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[48 (1436923539181535232)]} 0 0
[junit4:junit4]   2> 283569 T655 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[48 (1436923539186778112)]} 0 0
[junit4:junit4]   2> 283570 T621 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[48 (1436923539186778112)]} 0 4
[junit4:junit4]   2> 283570 T636 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[48]} 0 6
[junit4:junit4]   2> 283573 T601 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[49 (1436923539194118144)]} 0 0
[junit4:junit4]   2> 283579 T657 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[49 (1436923539197263872)]} 0 0
[junit4:junit4]   2> 283580 T623 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[49 (1436923539197263872)]} 0 4
[junit4:junit4]   2> 283582 T604 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[50 (1436923539203555328)]} 0 0
[junit4:junit4]   2> 283588 T658 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[50 (1436923539206701056)]} 0 0
[junit4:junit4]   2> 283589 T622 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[50 (1436923539206701056)]} 0 4
[junit4:junit4]   2> 283592 T599 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[51 (1436923539214041088)]} 0 0
[junit4:junit4]   2> 283598 T653 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[51 (1436923539217186816)]} 0 0
[junit4:junit4]   2> 283599 T625 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[51 (1436923539217186816)]} 0 4
[junit4:junit4]   2> 283603 T606 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[52 (1436923539225575424)]} 0 0
[junit4:junit4]   2> 283608 T654 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[52 (1436923539228721152)]} 0 0
[junit4:junit4]   2> 283610 T624 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[52 (1436923539228721152)]} 0 4
[junit4:junit4]   2> 283613 T605 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[53 (1436923539236061184)]} 0 0
[junit4:junit4]   2> 283621 T674 C1912 P59883 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[53 (1436923539240255488)]} 0 1
[junit4:junit4]   2> 283622 T641 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[53 (1436923539240255488)]} 0 5
[junit4:junit4]   2> 283625 T603 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[54 (1436923539248644096)]} 0 1
[junit4:junit4]   2> 283634 T656 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[54 (1436923539254935552)]} 0 0
[junit4:junit4]   2> 283635 T626 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[54 (1436923539254935552)]} 0 4
[junit4:junit4]   2> 283636 T639 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[54]} 0 7
[junit4:junit4]   2> 283638 T601 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[55 (1436923539262275584)]} 0 0
[junit4:junit4]   2> 283647 T655 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[55 (1436923539268567040)]} 0 0
[junit4:junit4]   2> 283648 T621 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[55 (1436923539268567040)]} 0 4
[junit4:junit4]   2> 283649 T640 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[55]} 0 8
[junit4:junit4]   2> 283651 T604 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[56 (1436923539275907072)]} 0 0
[junit4:junit4]   2> 283661 T670 C1912 P59883 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[56 (1436923539281149952)]} 0 1
[junit4:junit4]   2> 283662 T642 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[56 (1436923539281149952)]} 0 6
[junit4:junit4]   2> 283663 T623 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[56]} 0 9
[junit4:junit4]   2> 283666 T599 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[57 (1436923539291635712)]} 0 0
[junit4:junit4]   2> 283674 T675 C1912 P59883 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[57 (1436923539295830016)]} 0 1
[junit4:junit4]   2> 283675 T638 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[57 (1436923539295830016)]} 0 5
[junit4:junit4]   2> 283676 T622 C1911 P59855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[57]} 0 8
[junit4:junit4]   2> 283678 T606 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[58 (1436923539304218624)]} 0 0
[junit4:junit4]   2> 283686 T671 C1912 P59883 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59864/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[58 (1436923539307364352)]} 0 2
[junit4:junit4]   2> 283687 T636 C1913 P59864 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[58 (1436923539307364352)]} 0 6
[junit4:junit4]   2> 283689 T605 C1914 P59843 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={wt=javabin&version=2} {add=[59 (1436923539315752960)]} 0 0
[junit4:junit4]   2> 283696 T657 C1910 P59873 oasup.LogUpdateProcessor.finish [collection1] webapp=/ho_z/me path=/update params={distrib.from=http://127.0.0.1:59855/ho_z/me/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[59 (1436923539320995840)]} 0 0
[junit4:junit4]   2> 283697 T625 C1911 P59855 oasup.Log

[...truncated too long message...]

onverter
[junit4:junit4]   2> 1210491 T4159 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 1210492 T4159 oasss.Suggester.init init: {name=suggest_tst,classname=org.apache.solr.spelling.suggest.Suggester,lookupImpl=org.apache.solr.spelling.suggest.tst.TSTLookup,field=suggest,storeDir=suggest_tst,buildOnCommit=true,threshold=0.0}
[junit4:junit4]   2> 1210494 T4159 oashc.SpellCheckComponent.inform Registering newSearcher listener for spellchecker: suggest_tst
[junit4:junit4]   2> 1210495 T4159 oashc.SpellCheckComponent.inform No queryConverter defined, using default converter
[junit4:junit4]   2> 1210495 T4159 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 1210496 T4159 oasss.Suggester.init init: {name=suggest_fst,classname=org.apache.solr.spelling.suggest.Suggester,lookupImpl=org.apache.solr.spelling.suggest.fst.FSTLookup,field=suggest,storeDir=suggest_fst,buildOnCommit=true,weightBuckets=5,exactMatchFirst=true}
[junit4:junit4]   2> 1210498 T4159 oashc.SpellCheckComponent.inform Registering newSearcher listener for spellchecker: suggest_fst
[junit4:junit4]   2> 1210498 T4159 oashc.SpellCheckComponent.inform No queryConverter defined, using default converter
[junit4:junit4]   2> 1210498 T4159 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 1210498 T4159 oasss.Suggester.init init: {name=suggest_wfst,classname=org.apache.solr.spelling.suggest.Suggester,lookupImpl=org.apache.solr.spelling.suggest.fst.WFSTLookupFactory,field=suggest,storeDir=suggest_wfst,buildOnCommit=true,exactMatchFirst=true}
[junit4:junit4]   2> 1210502 T4159 oashc.SpellCheckComponent.inform Registering newSearcher listener for spellchecker: suggest_wfst
[junit4:junit4]   2> 1210502 T4159 oashc.SpellCheckComponent.inform No queryConverter defined, using default converter
[junit4:junit4]   2> 1210503 T4159 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 1210503 T4161 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@11ddd5 main{StandardDirectoryReader(segments_5:13 _2(5.0):C1 _3(5.0):C3)}
[junit4:junit4]   2> 1210512 T4159 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4:junit4]   2> ASYNC  NEW_CORE C2294 name=collection1 org.apache.solr.core.SolrCore@160cee6
[junit4:junit4]   2> 1210514 T4159 C2294 oasc.SolrCore.execute [collection1] webapp=null path=null params={q=ac&spellcheck.count=2&qt=/suggest_tst&spellcheck.onlyMorePopular=true} status=0 QTime=0 
[junit4:junit4]   2> 1210563 T4159 oas.SolrTestCaseJ4.tearDown ###Ending testReload
[junit4:junit4]   2> 1210565 T4159 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> 1210566 T4159 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1059305
[junit4:junit4]   2> 1210566 T4159 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@160cee6
[junit4:junit4]   2> 1210566 T4159 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4]   2> 1210566 T4159 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 1210566 T4159 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 1210567 T4159 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 1210568 T4159 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 1210569 T4159 oasc.CachingDirectoryFactory.close Closing MockFSDirectoryFactory - 2 directories currently being tracked
[junit4:junit4]   2> 1210569 T4159 oasc.CachingDirectoryFactory.closeCacheValue looking to close C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\solrtest-SuggesterTSTTest-1370357998828\index [CachedDir<<refCount=0;path=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\solrtest-SuggesterTSTTest-1370357998828\index;done=false>>]
[junit4:junit4]   2> 1210569 T4159 oasc.CachingDirectoryFactory.close Closing directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\solrtest-SuggesterTSTTest-1370357998828\index
[junit4:junit4]   2> 1210586 T4159 oasc.CachingDirectoryFactory.closeCacheValue looking to close C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\solrtest-SuggesterTSTTest-1370357998828 [CachedDir<<refCount=0;path=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\solrtest-SuggesterTSTTest-1370357998828;done=false>>]
[junit4:junit4]   2> 1210586 T4159 oasc.CachingDirectoryFactory.close Closing directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\solrtest-SuggesterTSTTest-1370357998828
[junit4:junit4]   2> NOTE: test params are: codec=DummyCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=DUMMY, chunkSize=147), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=DUMMY, chunkSize=147)), sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=ar_EG, timezone=Asia/Yekaterinburg
[junit4:junit4]   2> NOTE: Windows 7 6.1 x86/Oracle Corporation 1.7.0_21 (32-bit)/cpus=2,threads=1,free=416865656,total=536870912
[junit4:junit4]   2> NOTE: All tests run in this JVM: [SuggesterWFSTTest, DistributedSpellCheckComponentTest, NumericFieldsTest, TestQuerySenderListener, UnloadDistributedZkTest, OpenCloseCoreStressTest, ChaosMonkeyNothingIsSafeTest, TestAtomicUpdateErrorCases, SolrCoreCheckLockOnStartupTest, FieldAnalysisRequestHandlerTest, TestSolr4Spatial, DocValuesTest, TestSolrQueryParserDefaultOperatorResource, ResourceLoaderTest, ChaosMonkeyShardSplitTest, TestSchemaResource, IndexReaderFactoryTest, BinaryUpdateRequestHandlerTest, LeaderElectionIntegrationTest, SynonymTokenizerTest, EchoParamsTest, AutoCommitTest, TestUpdate, FastVectorHighlighterTest, SampleTest, TestCharFilters, DateFieldTest, TestIndexingPerformance, TestWordDelimiterFilterFactory, TestComponentsName, RequiredFieldsTest, SuggesterFSTTest, TestFuzzyAnalyzedSuggestions, FullSolrCloudDistribCmdsTest, ClusterStateTest, TestReplicationHandler, FieldMutatingUpdateProcessorTest, DirectUpdateHandlerOptimizeTest, RecoveryZkTest, TestDistributedSearch, SpellingQueryConverterTest, TestCSVLoader, TestSchemaSimilarityResource, TestPseudoReturnFields, SuggesterTest, TestLMDirichletSimilarityFactory, AlternateDirectoryTest, DocumentBuilderTest, DirectSolrConnectionTest, OpenExchangeRatesOrgProviderTest, TestQuerySenderNoQuery, ChaosMonkeySafeLeaderTest, TestMultiCoreConfBootstrap, SolrCmdDistributorTest, DocValuesMultiTest, XmlUpdateRequestHandlerTest, SolrIndexConfigTest, HighlighterConfigTest, URLClassifyProcessorTest, ShowFileRequestHandlerTest, BasicFunctionalityTest, TestIndexSearcher, LoggingHandlerTest, StatsComponentTest, ReturnFieldsTest, TestJmxMonitoredMap, PluginInfoTest, RegexBoostProcessorTest, TestStressRecovery, ClusterStateUpdateTest, TestSolrIndexConfig, TestCloudManagedSchema, TestCloudManagedSchemaAddField, ExternalFileFieldSortTest, IndexSchemaTest, IndexSchemaRuntimeFieldTest, TestSweetSpotSimilarityFactory, DistributedTermsComponentTest, TestFastOutputStream, TestSchemaNameResource, TestAddFieldRealTimeGet, AnalysisAfterCoreReloadTest, ShardRoutingTest, MoreLikeThisHandlerTest, BadComponentTest, UpdateParamsTest, TestManagedSchemaFieldResource, PathHierarchyTokenizerFactoryTest, TestLMJelinekMercerSimilarityFactory, CoreContainerCoreInitFailuresTest, HardAutoCommitTest, ZkNodePropsTest, ZkControllerTest, MultiTermTest, NotRequiredUniqueKeyTest, MBeansHandlerTest, SystemInfoHandlerTest, TestFieldTypeResource, QueryEqualityTest, TestSolrCoreProperties, TestSurroundQueryParser, SimpleFacetsTest, SchemaVersionSpecificBehaviorTest, SignatureUpdateProcessorFactoryTest, QueryParsingTest, TestFieldTypeCollectionResource, FileUtilsTest, SpatialFilterTest, TestSolrQueryParser, AliasIntegrationTest, ZkCLITest, LeaderElectionTest, TestHashPartitioner, DistributedQueryElevationComponentTest, JSONWriterTest, TestRTGBase, CSVRequestHandlerTest, RequestHandlersTest, TestSolrQueryParserResource, TestUniqueKeyFieldResource, TestReload, TestBadConfig, TestAnalyzedSuggestions, TestMergePolicyConfig, SuggesterTSTTest]
[junit4:junit4] Completed in 2.23s, 4 tests, 1 error <<< FAILURES!

[...truncated 526 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:380: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:360: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:39: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build.xml:181: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\common-build.xml:437: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1240: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:884: There were test failures: 296 suites, 1235 tests, 2 errors, 18 ignored (12 assumptions)

Total time: 66 minutes 55 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.7.0_21 -server -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-trunk-Windows (32bit/jdk1.7.0_21) - Build # 2892 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/2892/
Java: 32bit/jdk1.7.0_21 -client -XX:+UseConcMarkSweepGC

3 tests failed.
REGRESSION:  org.apache.solr.cloud.ChaosMonkeyShardSplitTest.testDistribSearch

Error Message:
Server at http://127.0.0.1:62993 returned non ok status:500, message:Server Error

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:62993 returned non ok status:500, message:Server Error
	at __randomizedtesting.SeedInfo.seed([E4C1C47DC3F0BC1C:65274A65B4AFDC20]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
	at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
	at org.apache.solr.cloud.ChaosMonkeyShardSplitTest.doTest(ChaosMonkeyShardSplitTest.java:136)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
	at sun.reflect.GeneratedMethodAccessor55.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)


REGRESSION:  org.apache.solr.cloud.ShardSplitTest.testDistribSearch

Error Message:
Server at http://127.0.0.1:63112 returned non ok status:500, message:Server Error

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:63112 returned non ok status:500, message:Server Error
	at __randomizedtesting.SeedInfo.seed([E4C1C47DC3F0BC1C:65274A65B4AFDC20]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
	at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
	at org.apache.solr.cloud.ShardSplitTest.doTest(ShardSplitTest.java:150)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
	at sun.reflect.GeneratedMethodAccessor55.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch

Error Message:
collection already exists: awholynewcollection_0

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: collection already exists: awholynewcollection_0
	at __randomizedtesting.SeedInfo.seed([E4C1C47DC3F0BC1C:65274A65B4AFDC20]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:424)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:264)
	at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:306)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1400)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:438)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:146)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)




Build Log:
[...truncated 9166 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
[junit4:junit4]   2> 342388 T564 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4]   2> 342392 T564 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\solrtest-CollectionsAPIDistributedZkTest-1370409576456
[junit4:junit4]   2> 342396 T564 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 342397 T565 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 342513 T564 oasc.ZkTestServer.run start zk server on port:59821
[junit4:junit4]   2> 342518 T564 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 342553 T571 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1693ce1 name:ZooKeeperConnection Watcher:127.0.0.1:59821 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 342554 T564 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 342554 T564 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 342571 T566 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13f12c6a8110000, likely client has closed socket
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 
[junit4:junit4]   2> 342576 T564 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 342585 T573 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@107bdd2 name:ZooKeeperConnection Watcher:127.0.0.1:59821/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 342586 T564 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 342586 T564 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 342603 T564 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 342615 T564 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 342625 T564 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 342634 T564 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 342636 T564 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 342653 T564 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 342653 T564 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 342660 T564 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 342661 T564 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 342668 T564 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 342669 T564 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 342676 T564 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 342677 T564 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 342684 T564 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 342685 T564 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 342691 T564 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 342691 T564 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 342699 T564 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 342700 T564 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 342709 T564 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 342710 T564 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 342721 T566 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13f12c6a8110001, likely client has closed socket
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 
[junit4:junit4]   2> 343213 T564 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 343223 T564 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:59828
[junit4:junit4]   2> 343223 T564 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 343224 T564 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 343225 T564 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370409576781
[junit4:junit4]   2> 343225 T564 oasc.CoreContainer$Initializer.initialize looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370409576781\solr.xml
[junit4:junit4]   2> 343225 T564 oasc.CoreContainer.<init> New CoreContainer 8735394
[junit4:junit4]   2> 343226 T564 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370409576781\'
[junit4:junit4]   2> 343226 T564 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370409576781\'
[junit4:junit4]   2> 343399 T564 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370409576781
[junit4:junit4]   2> 343403 T564 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370409576781\collection1
[junit4:junit4]   2> 343405 T564 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370409576781\collection1\conf
[junit4:junit4]   2> 343673 T564 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370409576781\collection1\conf\xslt
[junit4:junit4]   2> 343682 T564 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370409576781\collection1\lib
[junit4:junit4]   2> 343686 T564 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370409576781\collection1\lib\classes
[junit4:junit4]   2> 343694 T564 oasc.SolrCoreDiscoverer.addCore Discovered properties file C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370409576781\conf\core.properties, adding to cores
[junit4:junit4]   2> 343717 T564 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 343718 T564 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 343718 T564 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 343719 T564 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 343719 T564 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 343719 T564 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 343720 T564 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 343721 T564 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 343721 T564 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 343722 T564 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 343732 T564 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 343734 T564 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:59821/solr
[junit4:junit4]   2> 343734 T564 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 343736 T564 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 343754 T584 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@121484b name:ZooKeeperConnection Watcher:127.0.0.1:59821 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 343755 T564 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 343761 T564 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 343774 T564 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 343786 T586 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f9aeed name:ZooKeeperConnection Watcher:127.0.0.1:59821/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 343786 T564 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 343791 T564 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4]   2> 343800 T564 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4]   2> 343807 T564 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 343810 T564 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59828_
[junit4:junit4]   2> 343813 T564 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59828_
[junit4:junit4]   2> 343820 T564 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 343836 T564 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 343844 T564 oasc.Overseer.start Overseer (id=89811162003210243-127.0.0.1:59828_-n_0000000000) starting
[junit4:junit4]   2> 343856 T564 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4]   2> 343864 T588 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 343866 T564 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 343873 T564 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4]   2> 343877 T564 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 343883 T587 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 343885 T564 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
[junit4:junit4]   2> 343886 T564 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 343886 T564 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 343892 T564 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 343896 T564 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 343900 T591 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11c6c0f name:ZooKeeperConnection Watcher:127.0.0.1:59821/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 343900 T564 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 343904 T564 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 343907 T564 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 344418 T564 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 344424 T564 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:59840
[junit4:junit4]   2> 344425 T564 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 344426 T564 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 344426 T564 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370409577968
[junit4:junit4]   2> 344427 T564 oasc.CoreContainer$Initializer.initialize looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370409577968\solr.xml
[junit4:junit4]   2> 344427 T564 oasc.CoreContainer.<init> New CoreContainer 9357151
[junit4:junit4]   2> 344427 T564 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370409577968\'
[junit4:junit4]   2> 344428 T564 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370409577968\'
[junit4:junit4]   2> 344595 T564 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370409577968
[junit4:junit4]   2> 344597 T564 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370409577968\collection1
[junit4:junit4]   2> 344599 T564 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370409577968\collection1\conf
[junit4:junit4]   2> 344875 T564 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370409577968\collection1\conf\xslt
[junit4:junit4]   2> 344884 T564 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370409577968\collection1\lib
[junit4:junit4]   2> 344887 T564 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370409577968\collection1\lib\classes
[junit4:junit4]   2> 344894 T564 oasc.SolrCoreDiscoverer.addCore Discovered properties file C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370409577968\conf\core.properties, adding to cores
[junit4:junit4]   2> 344915 T564 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 344915 T564 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 344915 T564 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 344916 T564 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 344916 T564 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 344916 T564 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 344916 T564 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 344918 T564 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 344918 T564 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 344919 T564 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 344929 T564 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 344929 T564 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:59821/solr
[junit4:junit4]   2> 344930 T564 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 344932 T564 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 344952 T602 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d31d8f name:ZooKeeperConnection Watcher:127.0.0.1:59821 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 344953 T564 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 344957 T564 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 344965 T564 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 344970 T604 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@166149b name:ZooKeeperConnection Watcher:127.0.0.1:59821/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 344971 T564 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 344983 T564 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 346156 T564 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59840_
[junit4:junit4]   2> 346167 T564 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59840_
[junit4:junit4]   2> 346174 T591 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 346177 T586 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 346177 T604 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 346196 T564 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
[junit4:junit4]   2> 346196 T564 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 346197 T564 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 346698 T564 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 346706 T564 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:59849
[junit4:junit4]   2> 346708 T564 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 346708 T564 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 346708 T564 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370409580264
[junit4:junit4]   2> 346708 T564 oasc.CoreContainer$Initializer.initialize looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370409580264\solr.xml
[junit4:junit4]   2> 346709 T564 oasc.CoreContainer.<init> New CoreContainer 1250948
[junit4:junit4]   2> 346709 T564 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370409580264\'
[junit4:junit4]   2> 346709 T564 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370409580264\'
[junit4:junit4]   2> 346874 T564 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370409580264
[junit4:junit4]   2> 346876 T564 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370409580264\collection1
[junit4:junit4]   2> 346878 T564 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370409580264\collection1\conf
[junit4:junit4]   2> 347139 T564 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370409580264\collection1\conf\xslt
[junit4:junit4]   2> 347147 T564 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370409580264\collection1\lib
[junit4:junit4]   2> 347151 T564 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370409580264\collection1\lib\classes
[junit4:junit4]   2> 347157 T564 oasc.SolrCoreDiscoverer.addCore Discovered properties file C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370409580264\conf\core.properties, adding to cores
[junit4:junit4]   2> 347176 T564 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 347176 T564 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 347177 T564 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 347177 T564 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 347178 T564 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 347178 T564 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 347179 T564 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 347179 T564 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 347181 T564 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 347181 T564 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 347190 T564 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 347190 T564 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:59821/solr
[junit4:junit4]   2> 347191 T564 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 347194 T564 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 347212 T616 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15fe218 name:ZooKeeperConnection Watcher:127.0.0.1:59821 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 347214 T564 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 347219 T564 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 347247 T564 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 347255 T618 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e6f198 name:ZooKeeperConnection Watcher:127.0.0.1:59821/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 347256 T564 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 347276 T564 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 348540 T564 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59849_
[junit4:junit4]   2> 348549 T564 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59849_
[junit4:junit4]   2> 348560 T604 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 348561 T586 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 348563 T591 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 348563 T618 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 348575 T564 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
[junit4:junit4]   2> 348575 T564 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 348576 T564 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 349175 T564 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 349186 T564 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:59858
[junit4:junit4]   2> 349187 T564 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 349187 T564 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 349187 T564 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370409582644
[junit4:junit4]   2> 349187 T564 oasc.CoreContainer$Initializer.initialize looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370409582644\solr.xml
[junit4:junit4]   2> 349189 T564 oasc.CoreContainer.<init> New CoreContainer 28660186
[junit4:junit4]   2> 349189 T564 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370409582644\'
[junit4:junit4]   2> 349189 T564 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370409582644\'
[junit4:junit4]   2> 349378 T564 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370409582644
[junit4:junit4]   2> 349382 T564 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370409582644\collection1
[junit4:junit4]   2> 349384 T564 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370409582644\collection1\conf
[junit4:junit4]   2> 349658 T564 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370409582644\collection1\conf\xslt
[junit4:junit4]   2> 349668 T564 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370409582644\collection1\lib
[junit4:junit4]   2> 349670 T564 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370409582644\collection1\lib\classes
[junit4:junit4]   2> 349678 T564 oasc.SolrCoreDiscoverer.addCore Discovered properties file C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370409582644\conf\core.properties, adding to cores
[junit4:junit4]   2> 349708 T564 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 349708 T564 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 349709 T564 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 349709 T564 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 349710 T564 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 349710 T564 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 349710 T564 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 349712 T564 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 349712 T564 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 349712 T564 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 349723 T564 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 349724 T564 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:59821/solr
[junit4:junit4]   2> 349724 T564 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 349727 T564 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 349753 T630 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1815ddf name:ZooKeeperConnection Watcher:127.0.0.1:59821 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 349754 T564 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 349762 T564 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 349774 T564 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 349779 T632 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b6fd9c name:ZooKeeperConnection Watcher:127.0.0.1:59821/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 349781 T564 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 349791 T564 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 351083 T564 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59858_
[junit4:junit4]   2> 351099 T564 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59858_
[junit4:junit4]   2> 351117 T591 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 351118 T604 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 351119 T586 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 351121 T618 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 351122 T632 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 351146 T564 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
[junit4:junit4]   2> 351147 T564 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 351148 T564 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 351740 T564 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 351747 T564 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:59867
[junit4:junit4]   2> 351749 T564 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 351750 T564 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 351750 T564 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226
[junit4:junit4]   2> 351752 T564 oasc.CoreContainer$Initializer.initialize looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226\solr.xml
[junit4:junit4]   2> 351753 T564 oasc.CoreContainer.<init> New CoreContainer 28334968
[junit4:junit4]   2> 351753 T564 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226\'
[junit4:junit4]   2> 351754 T564 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226\'
[junit4:junit4]   2> 351959 T564 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226
[junit4:junit4]   2> 351961 T564 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226\collection1
[junit4:junit4]   2> 351962 T564 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226\collection1\conf
[junit4:junit4]   2> 352250 T564 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226\collection1\conf\xslt
[junit4:junit4]   2> 352259 T564 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226\collection1\lib
[junit4:junit4]   2> 352262 T564 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226\collection1\lib\classes
[junit4:junit4]   2> 352272 T564 oasc.SolrCoreDiscoverer.addCore Discovered properties file C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226\conf\core.properties, adding to cores
[junit4:junit4]   2> 352299 T564 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 352299 T564 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 352300 T564 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 352300 T564 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 352301 T564 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 352301 T564 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 352301 T564 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 352302 T564 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 352302 T564 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 352304 T564 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 352314 T564 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 352315 T564 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:59821/solr
[junit4:junit4]   2> 352315 T564 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 352319 T564 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 352340 T644 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f68ee8 name:ZooKeeperConnection Watcher:127.0.0.1:59821 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 352341 T564 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 352349 T564 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 352363 T564 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 352375 T646 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ef07c1 name:ZooKeeperConnection Watcher:127.0.0.1:59821/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 352377 T564 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 352388 T564 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 353678 T564 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59867_
[junit4:junit4]   2> 353699 T564 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59867_
[junit4:junit4]   2> 353709 T591 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 353711 T632 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 353711 T646 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 353713 T618 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 353713 T604 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 353714 T586 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 353729 T564 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
[junit4:junit4]   2> 353730 T564 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 353730 T564 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 353742 T564 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 353757 T595 oasha.CollectionsHandler.handleCreateAction Creating Collection : numShards=2&name=nodes_used_collection&replicationFactor=2&action=CREATE&wt=javabin&version=2
[junit4:junit4]   2> 353762 T586 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4]   2> 353765 T586 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4]   2> 353765 T588 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
[junit4:junit4]   2> 	  "operation":"createcollection",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "maxShardsPerNode":null,
[junit4:junit4]   2> 	  "createNodeSet":null,
[junit4:junit4]   2> 	  "name":"nodes_used_collection",
[junit4:junit4]   2> 	  "replicationFactor":"2"}
[junit4:junit4]   2> 353767 T588 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard1_replica1 as part of slice shard1 of collection nodes_used_collection on 127.0.0.1:59828_
[junit4:junit4]   2> 353768 T588 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard1_replica2 as part of slice shard1 of collection nodes_used_collection on 127.0.0.1:59849_
[junit4:junit4]   2> 353768 T588 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard2_replica1 as part of slice shard2 of collection nodes_used_collection on 127.0.0.1:59867_
[junit4:junit4]   2> 353770 T588 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard2_replica2 as part of slice shard2 of collection nodes_used_collection on 127.0.0.1:59858_
[junit4:junit4]   2> 353798 T610 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard1_replica2' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370409580264\nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 353799 T610 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4]   2> 353800 T625 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard2_replica2' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370409582644\nodes_used_collection_shard2_replica2
[junit4:junit4]   2> 353800 T625 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4]   2> 353799 T640 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard2_replica1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226\nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 353802 T640 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4]   2> 353802 T580 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard1_replica1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370409576781\nodes_used_collection_shard1_replica1
[junit4:junit4]   2> 353803 T580 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4]   2> 353803 T610 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4]   2> 353813 T610 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 353803 T625 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4]   2> 353814 T625 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 353814 T580 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4]   2> 353816 T580 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 353803 T640 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4]   2> 353817 T640 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 353819 T610 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 353819 T610 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4]   2> 353821 T625 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 353821 T625 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4]   2> 353822 T580 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 353821 T640 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 353823 T580 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4]   2> 353823 T640 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4]   2> 353836 T625 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4]   2> 353837 T610 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4]   2> 353839 T580 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4]   2> 353842 T640 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4]   2> 353846 T625 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370409582644\nodes_used_collection_shard2_replica2\'
[junit4:junit4]   2> 353847 T610 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370409580264\nodes_used_collection_shard1_replica2\'
[junit4:junit4]   2> 353849 T580 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370409576781\nodes_used_collection_shard1_replica1\'
[junit4:junit4]   2> 353852 T640 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226\nodes_used_collection_shard2_replica1\'
[junit4:junit4]   2> 353994 T625 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 354066 T610 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 354240 T625 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 354252 T640 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 354346 T625 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 354399 T625 oass.IndexSchema.readSchema [nodes_used_collection_shard2_replica2] Schema name=test
[junit4:junit4]   2> 354434 T640 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 354452 T610 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 354488 T580 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 354591 T640 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 354598 T640 oass.IndexSchema.readSchema [nodes_used_collection_shard2_replica1] Schema name=test
[junit4:junit4]   2> 354679 T610 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 354787 T610 oass.IndexSchema.readSchema [nodes_used_collection_shard1_replica2] Schema name=test
[junit4:junit4]   2> 354838 T580 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 355038 T580 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 355069 T580 oass.IndexSchema.readSchema [nodes_used_collection_shard1_replica1] Schema name=test
[junit4:junit4]   2> 357640 T625 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 357700 T625 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 357705 T625 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 358120 T640 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 358140 T640 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 358151 T640 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 358282 T610 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 358303 T610 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 358321 T610 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 358347 T625 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 358362 T625 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 358371 T625 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 358382 T625 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 358384 T625 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 358384 T625 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 358395 T625 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 358395 T625 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 358395 T625 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 358396 T625 oasc.SolrCore.<init> [nodes_used_collection_shard2_replica2] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370409582644\nodes_used_collection_shard2_replica2\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370409582644\nodes_used_collection_shard2_replica2\data\
[junit4:junit4]   2> 358401 T625 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@c2ee77
[junit4:junit4]   2> 358407 T625 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 358409 T625 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370409582644\nodes_used_collection_shard2_replica2\data\
[junit4:junit4]   2> 358409 T625 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370409582644\nodes_used_collection_shard2_replica2\data\index/
[junit4:junit4]   2> 358409 T625 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard2_replica2] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370409582644\nodes_used_collection_shard2_replica2\data\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 358409 T625 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370409582644\nodes_used_collection_shard2_replica2\data\index
[junit4:junit4]   2> 358415 T625 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1120709 lockFactory=org.apache.lucene.store.NativeFSLockFactory@f20092),segFN=segments_1,generation=1}
[junit4:junit4]   2> 358416 T625 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 358421 T625 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 358421 T625 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 358422 T625 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 358424 T625 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 358424 T625 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 358424 T625 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 358432 T625 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 358434 T625 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 358435 T625 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 358441 T580 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 358449 T625 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 358451 T640 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 358454 T625 oass.SolrIndexSearcher.<init> Opening Searcher@b9fc77 main
[junit4:junit4]   2> 358455 T625 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370409582644\nodes_used_collection_shard2_replica2\data\tlog
[junit4:junit4]   2> 358469 T580 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 358474 T625 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 358474 T625 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 358479 T580 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 358483 T640 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 358501 T640 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 358505 T652 oasc.SolrCore.registerSearcher [nodes_used_collection_shard2_replica2] Registered new searcher Searcher@b9fc77 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 358506 T625 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica2 state=down
[junit4:junit4]   2> 358508 T640 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 358514 T640 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 358514 T640 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 358509 T610 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 358523 T640 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 358523 T640 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 358525 T640 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 358525 T640 oasc.SolrCore.<init> [nodes_used_collection_shard2_replica1] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226\nodes_used_collection_shard2_replica1\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226\nodes_used_collection_shard2_replica1\data\
[junit4:junit4]   2> 358527 T640 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@c2ee77
[junit4:junit4]   2> 358527 T640 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 358528 T640 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226\nodes_used_collection_shard2_replica1\data\
[junit4:junit4]   2> 358528 T640 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226\nodes_used_collection_shard2_replica1\data\index/
[junit4:junit4]   2> 358528 T640 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard2_replica1] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226\nodes_used_collection_shard2_replica1\data\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 358528 T640 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226\nodes_used_collection_shard2_replica1\data\index
[junit4:junit4]   2> 358528 T610 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 358544 T640 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1891974 lockFactory=org.apache.lucene.store.NativeFSLockFactory@10ad99c),segFN=segments_1,generation=1}
[junit4:junit4]   2> 358544 T640 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 358549 T640 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 358549 T640 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 358550 T640 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 358550 T610 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 358552 T640 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 358553 T640 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 358553 T640 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 358554 T640 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 358554 T610 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 358555 T640 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 358555 T610 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 358558 T610 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 358555 T580 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 358555 T640 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 358563 T610 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 358563 T610 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 358563 T610 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 358564 T610 oasc.SolrCore.<init> [nodes_used_collection_shard1_replica2] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370409580264\nodes_used_collection_shard1_replica2\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370409580264\nodes_used_collection_shard1_replica2\data\
[junit4:junit4]   2> 358564 T610 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@c2ee77
[junit4:junit4]   2> 358565 T610 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 358570 T610 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370409580264\nodes_used_collection_shard1_replica2\data\
[junit4:junit4]   2> 358570 T610 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370409580264\nodes_used_collection_shard1_replica2\data\index/
[junit4:junit4]   2> 358572 T610 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard1_replica2] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370409580264\nodes_used_collection_shard1_replica2\data\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 358573 T610 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370409580264\nodes_used_collection_shard1_replica2\data\index
[junit4:junit4]   2> 358578 T610 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1732a6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@83f397),segFN=segments_1,generation=1}
[junit4:junit4]   2> 358579 T610 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 358584 T610 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 358584 T610 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 358586 T610 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 358593 T610 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 358594 T580 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 358594 T610 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 358594 T610 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 358596 T640 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 358596 T610 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 358597 T610 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 358597 T610 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 358601 T640 oass.SolrIndexSearcher.<init> Opening Searcher@ee38df main
[junit4:junit4]   2> 358602 T640 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226\nodes_used_collection_shard2_replica1\data\tlog
[junit4:junit4]   2> 358611 T610 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 358618 T640 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 358619 T640 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 358619 T580 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 358621 T610 oass.SolrIndexSearcher.<init> Opening Searcher@bb2118 main
[junit4:junit4]   2> 358637 T653 oasc.SolrCore.registerSearcher [nodes_used_collection_shard2_replica1] Registered new searcher Searcher@ee38df main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 358641 T640 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica1 state=down
[junit4:junit4]   2> 358643 T610 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370409580264\nodes_used_collection_shard1_replica2\data\tlog
[junit4:junit4]   2> 358646 T610 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 358646 T610 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 358655 T610 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica2 state=down
[junit4:junit4]   2> 358657 T654 oasc.SolrCore.registerSearcher [nodes_used_collection_shard1_replica2] Registered new searcher Searcher@bb2118 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 358658 T580 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 358658 T580 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 358658 T580 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 358665 T580 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 358666 T580 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 358666 T580 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 358666 T580 oasc.SolrCore.<init> [nodes_used_collection_shard1_replica1] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370409576781\nodes_used_collection_shard1_replica1\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370409576781\nodes_used_collection_shard1_replica1\data\
[junit4:junit4]   2> 358666 T580 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@c2ee77
[junit4:junit4]   2> 358667 T580 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 358668 T580 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370409576781\nodes_used_collection_shard1_replica1\data\
[junit4:junit4]   2> 358668 T580 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370409576781\nodes_used_collection_shard1_replica1\data\index/
[junit4:junit4]   2> 358670 T580 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard1_replica1] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370409576781\nodes_used_collection_shard1_replica1\data\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 358671 T580 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370409576781\nodes_used_collection_shard1_replica1\data\index
[junit4:junit4]   2> 358676 T580 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1199e96 lockFactory=org.apache.lucene.store.NativeFSLockFactory@18c6637),segFN=segments_1,generation=1}
[junit4:junit4]   2> 358676 T580 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 358688 T580 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 358688 T580 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 358690 T580 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 358690 T580 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 358691 T580 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 358691 T580 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 358691 T580 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 358692 T580 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 358693 T580 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 358707 T580 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 358731 T580 oass.SolrIndexSearcher.<init> Opening Searcher@a3c153 main
[junit4:junit4]   2> 358732 T580 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370409576781\nodes_used_collection_shard1_replica1\data\tlog
[junit4:junit4]   2> 358734 T580 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 358734 T580 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 358744 T580 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica1 state=down
[junit4:junit4]   2> 358747 T655 oasc.SolrCore.registerSearcher [nodes_used_collection_shard1_replica1] Registered new searcher Searcher@a3c153 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 359219 T587 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 359242 T587 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard2_replica2",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:59858_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:59858"}
[junit4:junit4]   2> 359243 T587 oasc.Overseer$ClusterStateUpdater.createCollection Create collection nodes_used_collection with numShards 2
[junit4:junit4]   2> 359273 T587 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard2_replica1",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:59867_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:59867"}
[junit4:junit4]   2> 359284 T587 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard1_replica2",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:59849_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:59849"}
[junit4:junit4]   2> 359293 T587 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard1_replica1",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:59828_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:59828"}
[junit4:junit4]   2> 359303 T646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 359304 T632 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 359304 T586 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 359304 T604 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 359306 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 359306 T591 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 359805 T625 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard2_replica2
[junit4:junit4]   2> 359806 T625 oasc.ZkController.register Register replica - core:nodes_used_collection_shard2_replica2 address:http://127.0.0.1:59858 collection:nodes_used_collection shard:shard2
[junit4:junit4]   2> 359809 T625 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leader_elect/shard2/election
[junit4:junit4]   2> 359855 T625 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 359871 T625 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up: total=2 found=1 timeoutin=179999
[junit4:junit4]   2> 359943 T640 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 359944 T640 oasc.ZkController.register Register replica - core:nodes_used_collection_shard2_replica1 address:http://127.0.0.1:59867 collection:nodes_used_collection shard:shard2
[junit4:junit4]   2> 359948 T610 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 359949 T610 oasc.ZkController.register Register replica - core:nodes_used_collection_shard1_replica2 address:http://127.0.0.1:59849 collection:nodes_used_collection shard:shard1
[junit4:junit4]   2> 359953 T610 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leader_elect/shard1/election
[junit4:junit4]   2> 359998 T610 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 360007 T610 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up: total=2 found=1 timeoutin=179999
[junit4:junit4]   2> 360033 T580 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard1_replica1
[junit4:junit4]   2> 360033 T580 oasc.ZkController.register Register replica - core:nodes_used_collection_shard1_replica1 address:http://127.0.0.1:59828 collection:nodes_used_collection shard:shard1
[junit4:junit4]   2> 360523 T625 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 360523 T625 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 360526 T625 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:59858/nodes_used_collection_shard2_replica2/
[junit4:junit4]   2> 360526 T625 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard2_replica2 url=http://127.0.0.1:59858 START replicas=[http://127.0.0.1:59867/nodes_used_collection_shard2_replica1/] nUpdates=100
[junit4:junit4]   2> 360527 T625 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard2_replica2 url=http://127.0.0.1:59858 DONE.  We have no versions.  sync failed.
[junit4:junit4]   2> 360527 T625 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next canidate
[junit4:junit4]   2> 360537 T625 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4:junit4]   2> 360538 T625 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:59858/nodes_used_collection_shard2_replica2/
[junit4:junit4]   2> 360538 T625 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leaders/shard2
[junit4:junit4]   2> ASYNC  NEW_CORE C9185 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@1d3057 url=http://127.0.0.1:59867/nodes_used_collection_shard2_replica1 node=127.0.0.1:59867_ C9185_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=down, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:59867_, base_url=http://127.0.0.1:59867}
[junit4:junit4]   2> 360574 T639 C9185 P59867 oasc.SolrCore.execute [nodes_used_collection_shard2_replica1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1 
[junit4:junit4]   2> 360674 T610 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 360674 T610 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 360675 T610 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:59849/nodes_used_collection_shard1_replica2/
[junit4:junit4]   2> 360675 T610 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard1_replica2 url=http://127.0.0.1:59849 START replicas=[http://127.0.0.1:59828/nodes_used_collection_shard1_replica1/] nUpdates=100
[junit4:junit4]   2> 360675 T610 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard1_replica2 url=http://127.0.0.1:59849 DONE.  We have no versions.  sync failed.
[junit4:junit4]   2> 360675 T610 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next canidate
[junit4:junit4]   2> 360675 T610 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4:junit4]   2> 360676 T610 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:59849/nodes_used_collection_shard1_replica2/
[junit4:junit4]   2> 360678 T610 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leaders/shard1
[junit4:junit4]   2> ASYNC  NEW_CORE C9186 name=nodes_used_collection_shard1_replica1 org.apache.solr.core.SolrCore@1984e73 url=http://127.0.0.1:59828/nodes_used_collection_shard1_replica1 node=127.0.0.1:59828_ C9186_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica1 props:{shard=shard1, state=down, core=nodes_used_collection_shard1_replica1, collection=nodes_used_collection, node_name=127.0.0.1:59828_, base_url=http://127.0.0.1:59828}
[junit4:junit4]   2> 360701 T578 C9186 P59828 oasc.SolrCore.execute [nodes_used_collection_shard1_replica1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=2 
[junit4:junit4]   2> 361342 T587 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 361398 T646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 361400 T591 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 361400 T632 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 361401 T604 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 361401 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 361400 T586 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 361430 T610 oasc.ZkController.register We are http://127.0.0.1:59849/nodes_used_collection_shard1_replica2/ and leader is http://127.0.0.1:59849/nodes_used_collection_shard1_replica2/
[junit4:junit4]   2> 361430 T610 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard1_replica2 baseURL=http://127.0.0.1:59849
[junit4:junit4]   2> 361431 T610 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 361431 T610 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica2 state=active
[junit4:junit4]   2> 361436 T610 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 361438 T610 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard1&name=nodes_used_collection_shard1_replica2&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=7641 
[junit4:junit4]   2> 361449 T625 oasc.ZkController.register We are http://127.0.0.1:59858/nodes_used_collection_shard2_replica2/ and leader is http://127.0.0.1:59858/nodes_used_collection_shard2_replica2/
[junit4:junit4]   2> 361449 T625 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard2_replica2 baseURL=http://127.0.0.1:59858
[junit4:junit4]   2> 361451 T625 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 361451 T625 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica2 state=active
[junit4:junit4]   2> 361456 T625 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 361459 T625 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard2&name=nodes_used_collection_shard2_replica2&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=7659 
[junit4:junit4]   2> 361469 T640 oasc.ZkController.register We are http://127.0.0.1:59867/nodes_used_collection_shard2_replica1/ and leader is http://127.0.0.1:59858/nodes_used_collection_shard2_replica2/
[junit4:junit4]   2> 361469 T640 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard2_replica1 baseURL=http://127.0.0.1:59867
[junit4:junit4]   2> 361469 T640 oasc.ZkController.checkRecovery Core needs to recover:nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 361470 T640 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C9187 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@1d3057 url=http://127.0.0.1:59867/nodes_used_collection_shard2_replica1 node=127.0.0.1:59867_ C9187_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=down, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:59867_, base_url=http://127.0.0.1:59867}
[junit4:junit4]   2> 361471 T658 C9187 P59867 oasc.RecoveryStrategy.run Starting recovery process.  core=nodes_used_collection_shard2_replica1 recoveringAfterStartup=true
[junit4:junit4]   2> 361471 T658 C9187 P59867 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 361473 T658 C9187 P59867 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica1 state=recovering
[junit4:junit4]   2> 361473 T640 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 361478 T658 C9187 P59867 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 361480 T640 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard2&name=nodes_used_collection_shard2_replica1&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=7681 
[junit4:junit4]   2> 361491 T580 oasc.ZkController.register We are http://127.0.0.1:59828/nodes_used_collection_shard1_replica1/ and leader is http://127.0.0.1:59849/nodes_used_collection_shard1_replica2/
[junit4:junit4]   2> 361491 T580 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard1_replica1 baseURL=http://127.0.0.1:59828
[junit4:junit4]   2> 361491 T580 oasc.ZkController.checkRecovery Core needs to recover:nodes_used_collection_shard1_replica1
[junit4:junit4]   2> 361491 T580 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> 361495 T626 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:59867__nodes_used_collection_shard2_replica1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4]   2> 361495 T580 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> ASYNC  NEW_CORE C9188 name=nodes_used_collection_shard1_replica1 org.apache.solr.core.SolrCore@1984e73 url=http://127.0.0.1:59828/nodes_used_collection_shard1_replica1 node=127.0.0.1:59828_ C9188_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica1 props:{shard=shard1, state=down, core=nodes_used_collection_shard1_replica1, collection=nodes_used_collection, node_name=127.0.0.1:59828_, base_url=http://127.0.0.1:59828}
[junit4:junit4]   2> 361495 T659 C9188 P59828 oasc.RecoveryStrategy.run Starting recovery process.  core=nodes_used_collection_shard1_replica1 recoveringAfterStartup=true
[junit4:junit4]   2> 361496 T659 C9188 P59828 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 361498 T659 C9188 P59828 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica1 state=recovering
[junit4:junit4]   2> 361498 T580 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard1&name=nodes_used_collection_shard1_replica1&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=7696 
[junit4:junit4]   2> 361500 T588 oasc.OverseerCollectionProcessor.createCollection Finished create command on all shards for collection: nodes_used_collection
[junit4:junit4]   2> 361502 T659 C9188 P59828 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 361520 T611 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:59828__nodes_used_collection_shard1_replica1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4]   2> 361524 T604 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work/qnr-0000000000 state: SyncConnected type NodeDataChanged
[junit4:junit4]   2> 361529 T588 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Message id:/overseer/collection-queue-work/qn-0000000000 complete, response:{success={null={responseHeader={status=0,QTime=7641},core=nodes_used_collection_shard1_replica2},null={responseHeader={status=0,QTime=7659},core=nodes_used_collection_shard2_replica2},null={responseHeader={status=0,QTime=7681},core=nodes_used_collection_shard2_replica1},null={responseHeader={status=0,QTime=7696},core=nodes_used_collection_shard1_replica1}}}
[junit4:junit4]   2> 361536 T595 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={numShards=2&name=nodes_used_collection&replicationFactor=2&action=CREATE&wt=javabin&version=2} status=0 QTime=7784 
[junit4:junit4]   2> 361539 T564 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 361542 T564 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 361549 T661 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@27288c name:ZooKeeperConnection Watcher:127.0.0.1:59821/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 361550 T564 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 361553 T564 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 361558 T564 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 361564 T564 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 361569 T663 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19cfa4a name:ZooKeeperConnection Watcher:127.0.0.1:59821/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 361569 T564 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 361574 T564 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 361608 T581 oasha.CollectionsHandler.handleCreateAction Creating Collection : numShards=5&maxShardsPerNode=6&name=awholynewcollection_0&replicationFactor=5&action=CREATE&wt=javabin&version=2
[junit4:junit4]   2> 361616 T586 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4]   2> 361622 T588 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000002 message:{
[junit4:junit4]   2> 	  "operation":"createcollection",
[junit4:junit4]   2> 	  "numShards":"5",
[junit4:junit4]   2> 	  "maxShardsPerNode":"6",
[junit4:junit4]   2> 	  "createNodeSet":null,
[junit4:junit4]   2> 	  "name":"awholynewcollection_0",
[junit4:junit4]   2> 	  "replicationFactor":"5"}
[junit4:junit4]   2> 361622 T588 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica1 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:59849_
[junit4:junit4]   2> 361622 T588 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica2 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:59840_
[junit4:junit4]   2> 361623 T588 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica3 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:59858_
[junit4:junit4]   2> 361623 T588 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica4 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:59867_
[junit4:junit4]   2> 361625 T588 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica5 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:59828_
[junit4:junit4]   2> 361627 T612 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370409580264\awholynewcollection_0_shard1_replica1
[junit4:junit4]   2> 361629 T612 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 361627 T596 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica2' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370409577968\awholynewcollection_0_shard1_replica2
[junit4:junit4]   2> 361630 T642 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica4' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226\awholynewcollection_0_shard1_replica4
[junit4:junit4]   2> 361630 T642 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 361629 T588 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard2_replica1 as part of slice shard2 of collection awholynewcollection_0 on 127.0.0.1:59849_
[junit4:junit4]   2> 361629 T627 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica3' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370409582644\awholynewcollection_0_shard1_replica3
[junit4:junit4]   2> 361633 T627 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 361633 T588 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard2_replica2 as part of slice shard2 of collection awholynewcollection_0 on 127.0.0.1:59840_
[junit4:junit4]   2> 361631 T612 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4]   2> 361634 T612 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 361636 T588 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard2_replica3 as part of slice shard2 of collection awholynewcollection_0 on 127.0.0.1:59858_
[junit4:junit4]   2> 361630 T596 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 361640 T614 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard2_replica1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370409580264\awholynewcollection_0_shard2_replica1
[junit4:junit4]   2> 361637 T588 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard2_replica4 as part of slice shard2 of collection awholynewcollection_0 on 127.0.0.1:59867_
[junit4:junit4]   2> 361641 T614 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 361643 T621 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard2_replica3' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370409582644\awholynewcollection_0_shard2_replica3
[junit4:junit4]   2> 361644 T621 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 361644 T612 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 361644 T612 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4]   2> 361634 T627 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4]   2> 361647 T621 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4]   2> 361648 T621 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 361634 T582 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica5' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370409576781\awholynewcollection_0_shard1_replica5
[junit4:junit4]   2> 361649 T582 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 361634 T642 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4]   2> 361651 T642 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 361652 T582 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4]   2> 361648 T627 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 361648 T638 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard2_replica4' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226\awholynewcollection_0_shard2_replica4
[junit4:junit4]   2> 361654 T638 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 361647 T614 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4]   2> 361655 T614 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 361645 T596 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4]   2> 361658 T596 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 361658 T638 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 361658 T638 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 361643 T597 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard2_replica2' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370409577968\awholynewcollection_0_shard2_replica2
[junit4:junit4]   2> 361660 T597 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 361662 T612 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 361662 T627 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4]   2> 361641 T588 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard2_replica5 as part of slice shard2 of collection awholynewcollection_0 on 127.0.0.1:59828_
[junit4:junit4]   2> 361662 T642 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4]   2> 361667 T597 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 361667 T597 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 361655 T621 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 361670 T621 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4]   2> 361652 T582 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 361670 T596 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4]   2> 361670 T638 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226\awholynewcollection_0_shard2_replica4\'
[junit4:junit4]   2> 361689 T612 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370409580264\awholynewcollection_0_shard1_replica1\'
[junit4:junit4]   2> 361691 T577 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard2_replica5' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370409576781\awholynewcollection_0_shard2_replica5
[junit4:junit4]   2> 361691 T577 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 361667 T614 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4]   2> 361667 T588 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard3_replica1 as part of slice shard3 of collection awholynewcollection_0 on 127.0.0.1:59849_
[junit4:junit4]   2> 361700 T588 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard3_replica2 as part of slice shard3 of collection awholynewcollection_0 on 127.0.0.1:59840_
[junit4:junit4]   2> 361702 T588 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard3_replica3 as part of slice shard3 of collection awholynewcollection_0 on 127.0.0.1:59858_
[junit4:junit4]   2> 361702 T588 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard3_replica4 as part of slice shard3 of collection awholynewcollection_0 on 127.0.0.1:59867_
[junit4:junit4]   2> 361707 T588 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard3_replica5 as part of slice shard3 of collection awholynewcollection_0 on 127.0.0.1:59828_
[junit4:junit4]   2> 361707 T588 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard4_replica1 as part of slice shard4 of collection awholynewcollection_0 on 127.0.0.1:59849_
[junit4:junit4]   2> 361712 T588 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard4_replica2 as part of slice shard4 of collection awholynewcollection_0 on 127.0.0.1:59840_
[junit4:junit4]   2> 361713 T588 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard4_replica3 as part of slice shard4 of collection awholynewcollection_0 on 127.0.0.1:59858_
[junit4:junit4]   2> 361714 T588 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard4_replica4 as part of slice shard4 of collection awholynewcollection_0 on 127.0.0.1:59867_
[junit4:junit4]   2> 361714 T588 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard4_replica5 as part of slice shard4 of collection awholynewcollection_0 on 127.0.0.1:59828_
[junit4:junit4]   2> 361720 T588 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard5_replica1 as part of slice shard5 of collection awholynewcollection_0 on 127.0.0.1:59849_
[junit4:junit4]   2> 361721 T588 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard5_replica2 as part of slice shard5 of collection awholynewcollection_0 on 127.0.0.1:59840_
[junit4:junit4]   2> 361724 T588 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard5_replica3 as part of slice shard5 of collection awholynewcollection_0 on 127.0.0.1:59858_
[junit4:junit4]   2> 361725 T588 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard5_replica4 as part of slice shard5 of collection awholynewcollection_0 on 127.0.0.1:59867_
[junit4:junit4]   2> 361727 T588 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard5_replica5 as part of slice shard5 of collection awholynewcollection_0 on 127.0.0.1:59828_
[junit4:junit4]   2> 361746 T577 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 361761 T577 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 361764 T597 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370409577968\awholynewcollection_0_shard2_replica2\'
[junit4:junit4]   2> 361758 T624 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard3_replica3' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370409582644\awholynewcollection_0_shard3_replica3
[junit4:junit4]   2> 361779 T624 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 361756 T609 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard3_replica1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370409580264\awholynewcollection_0_shard3_replica1
[junit4:junit4]   2> 361804 T609 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 361807 T621 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 361807 T627 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 361754 T642 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 361814 T624 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 361814 T624 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 361819 T596 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 361754 T639 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard5_replica4' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226\awholynewcollection_0_shard5_replica4
[junit4:junit4]   2> 361837 T639 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 361836 T609 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 361840 T609 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 361843 T621 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370409582644\awholynewcollection_0_shard2_replica3\'
[junit4:junit4]   2> 361836 T614 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 361793 T628 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard4_replica3' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370409582644\awholynewcollection_0_shard4_replica3
[junit4:junit4]   2> 361850 T628 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 361854 T627 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370409582644\awholynewcollection_0_shard1_replica3\'
[junit4:junit4]   2> 361793 T578 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard5_replica5' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370409576781\awholynewcollection_0_shard5_replica5
[junit4:junit4]   2> 361888 T578 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 361890 T642 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226\awholynewcollection_0_shard1_replica4\'
[junit4:junit4]   2> 361895 T628 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 361895 T628 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 361896 T624 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370409582644\awholynewcollection_0_shard3_replica3\'
[junit4:junit4]   2> 361793 T625 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard5_replica3' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370409582644\awholynewcollection_0_shard5_replica3
[junit4:junit4]   2> 361927 T625 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 361792 T580 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard4_replica5' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370409576781\awholynewcollection_0_shard4_replica5
[junit4:junit4]   2> 361930 T580 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 361935 T578 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 361935 T578 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 361936 T580 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 361936 T580 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 361939 T614 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370409580264\awholynewcollection_0_shard2_replica1\'
[junit4:junit4]   2> 361792 T579 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard3_replica5' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370409576781\awholynewcollection_0_shard3_replica5
[junit4:junit4]   2> 361970 T579 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 361977 T628 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370409582644\awholynewcollection_0_shard4_replica3\'
[junit4:junit4]   2> 361792 T595 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard5_replica2' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370409577968\awholynewcollection_0_shard5_replica2
[junit4:junit4]   2> 361981 T595 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 361985 T578 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370409576781\awholynewcollection_0_shard5_replica5\'
[junit4:junit4]   2> 361790 T599 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard3_replica2' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370409577968\awholynewcollection_0_shard3_replica2
[junit4:junit4]   2> 361986 T599 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 361790 T598 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard4_replica2' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370409577968\awholynewcollection_0_shard4_replica2
[junit4:junit4]   2> 361988 T595 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 361988 T595 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 361789 T613 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard4_replica1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370409580264\awholynewcollection_0_shard4_replica1
[junit4:junit4]   2> 361992 T613 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 361789 T610 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard5_replica1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370409580264\awholynewcollection_0_shard5_replica1
[junit4:junit4]   2> 361993 T610 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 361787 T640 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard3_replica4' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226\awholynewcollection_0_shard3_replica4
[junit4:junit4]   2> 362029 T640 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 361775 T641 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard4_replica4' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226\awholynewcollection_0_shard4_replica4
[junit4:junit4]   2> 362030 T641 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 362032 T599 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 362033 T599 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 361763 T582 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4]   2> 362034 T613 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 362034 T613 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 362033 T610 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 362036 T610 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 362000 T580 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370409576781\awholynewcollection_0_shard4_replica5\'
[junit4:junit4]   2> 362039 T641 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 362039 T641 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 361988 T598 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 361986 T579 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 362074 T579 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 361970 T625 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 362076 T625 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 361954 T596 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370409577968\awholynewcollection_0_shard1_replica2\'
[junit4:junit4]   2> 362084 T598 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 362084 T598 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 361952 T609 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370409580264\awholynewcollection_0_shard3_replica1\'
[junit4:junit4]   2> 361950 T612 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 361848 T639 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 362101 T639 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 361838 T577 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370409576781\awholynewcollection_0_shard2_replica5\'
[junit4:junit4]   2> 362099 T595 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370409577968\awholynewcollection_0_shard5_replica2\'
[junit4:junit4]   2> 362099 T599 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370409577968\awholynewcollection_0_shard3_replica2\'
[junit4:junit4]   2> 362036 T640 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 362113 T640 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 362117 T641 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226\awholynewcollection_0_shard4_replica4\'
[junit4:junit4]   2> 362124 T625 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370409582644\awholynewcollection_0_shard5_replica3\'
[junit4:junit4]   2> 362131 T610 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370409580264\awholynewcollection_0_shard5_replica1\'
[junit4:junit4]   2> 362131 T613 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370409580264\awholynewcollection_0_shard4_replica1\'
[junit4:junit4]   2> 362221 T579 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370409576781\awholynewcollection_0_shard3_replica5\'
[junit4:junit4]   2> 362230 T639 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226\awholynewcollection_0_shard5_replica4\'
[junit4:junit4]   2> 362244 T597 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 362262 T582 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 362290 T598 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370409577968\awholynewcollection_0_shard4_replica2\'
[junit4:junit4]   2> 362291 T640 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226\awholynewcollection_0_shard3_replica4\'
[junit4:junit4]   2> 362299 T582 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370409576781\awholynewcollection_0_shard1_replica5\'
[junit4:junit4]   2> 362480 T642 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 362546 T621 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 362798 T612 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 362846 T628 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 362878 T612 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 363101 T638 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 363217 T580 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 363352 T614 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 363370 T627 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 363377 T597 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 363379 T612 oass.IndexSchema.readSchema [awholynewcollection_0_shard1_replica1] Schema name=test
[junit4:junit4]   2> 363404 T597 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 363524 T624 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 363680 T580 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 363685 T587 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 363685 T580 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 363721 T587 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:59849__nodes_used_collection_shard1_replica2",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard1_replica2",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:59849_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:59849"}
[junit4:junit4]   2> 363838 T580 oass.IndexSchema.readSchema [awholynewcollection_0_shard4_replica5] Schema name=test
[junit4:junit4]   2> 363891 T597 oass.IndexSchema.readSchema [awholynewcollection_0_shard2_replica2] Schema name=test
[junit4:junit4]   2> 363949 T587 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:59858__nodes_used_collection_shard2_replica2",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard2_replica2",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:59858_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:59858"}
[junit4:junit4]   2> 364054 T642 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 364055 T577 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 364146 T578 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 364168 T642 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 364190 T587 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:59867__nodes_used_collection_shard2_replica1",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard2_replica1",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:59867_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:59867"}
[junit4:junit4]   2> 364316 T642 oass.IndexSchema.readSchema [awholynewcollection_0_shard1_replica4] Schema name=test
[junit4:junit4]   2> 364375 T587 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:59828__nodes_used_collection_shard1_replica1",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard1_replica1",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:59828_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:59828"}
[junit4:junit4]   2> 364520 T624 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 364590 T591 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 364592 T663 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 364593 T624 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 364601 T624 oass.IndexSchema.readSchema [awholynewcollection_0_shard3_replica3] Schema name=test
[junit4:junit4]   2> 364612 T632 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 364612 T604 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 364614 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 364614 T646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 364615 T661 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 364617 T586 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 364875 T582 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 364897 T609 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 365006 T621 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 365149 T614 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 365159 T621 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 365187 T640 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 365229 T614 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 365238 T614 oass.IndexSchema.readSchema [awholynewcollection_0_shard2_replica1] Schema name=test
[junit4:junit4]   2> 365476 T621 oass.IndexSchema.readSchema [awholynewcollection_0_shard2_replica3] Schema name=test
[junit4:junit4]   2> 365504 T638 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 365587 T639 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 365634 T638 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 365642 T638 oass.IndexSchema.readSchema [awholynewcollection_0_shard2_replica4] Schema name=test
[junit4:junit4]   2> 365735 T579 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 365777 T626 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:59867__nodes_used_collection_shard2_replica1, state: recovering, checkLive: true, onlyIfLeader: true for: 3 seconds.
[junit4:junit4]   2> 365779 T626 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:59867__nodes_used_collection_shard2_replica1&state=recovering&nodeName=127.0.0.1:59867_&action=PREPRECOVERY&checkLive=true&core=nodes_used_collection_shard2_replica2&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=4284 
[junit4:junit4]   2> 365806 T611 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:59828__nodes_used_collection_shard1_replica1, state: recovering, checkLive: true, onlyIfLeader: true for: 3 seconds.
[junit4:junit4]   2> 365808 T611 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:59828__nodes_used_collection_shard1_replica1&state=recovering&nodeName=127.0.0.1:59828_&action=PREPRECOVERY&checkLive=true&core=nodes_used_collection_shard1_replica2&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=4286 
[junit4:junit4]   2> 366109 T595 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 366197 T598 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 366220 T596 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 366300 T625 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 366438 T610 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 366574 T613 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 366627 T627 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 366658 T627 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 366666 T577 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 366712 T577 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 366975 T578 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 367051 T582 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 367064 T578 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 367068 T641 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 367120 T577 oass.IndexSchema.readSchema [awholynewcollection_0_shard2_replica5] Schema name=test
[junit4:junit4]   2> 367137 T582 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 367152 T627 oass.IndexSchema.readSchema [awholynewcollection_0_shard1_replica3] Schema name=test
[junit4:junit4]   2> 367273 T582 oass.IndexSchema.readSchema [awholynewcollection_0_shard1_replica5] Schema name=test
[junit4:junit4]   2> 367293 T578 oass.IndexSchema.readSchema [awholynewcollection_0_shard5_replica5] Schema name=test
[junit4:junit4]   2> 367486 T640 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 367553 T640 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 367582 T640 oass.IndexSchema.readSchema [awholynewcollection_0_shard3_replica4] Schema name=test
[junit4:junit4]   2> 367762 T598 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 367912 T598 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 367976 T599 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 368119 T609 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 368188 T610 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 368234 T609 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 368301 T609 oass.IndexSchema.readSchema [awholynewcollection_0_shard3_replica1] Schema name=test
[junit4:junit4]   2> 368317 T610 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 368416 T598 oass.IndexSchema.readSchema [awholynewcollection_0_shard4_replica2] Schema name=test
[junit4:junit4]   2> 368416 T610 oass.IndexSchema.readSchema [awholynewcollection_0_shard5_replica1] Schema name=test
[junit4:junit4]   2> ASYNC  NEW_CORE C9189 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@1d3057 url=http://127.0.0.1:59867/nodes_used_collection_shard2_replica1 node=127.0.0.1:59867_ C9189_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:59867_, base_url=http://127.0.0.1:59867}
[junit4:junit4]   2> 368609 T658 C9189 P59867 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:59858/nodes_used_collection_shard2_replica2/ core=nodes_used_collection_shard2_replica1 - recoveringAfterStartup=true
[junit4:junit4]   2> 368611 T658 C9189 P59867 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard2_replica1 url=http://127.0.0.1:59867 START replicas=[http://127.0.0.1:59858/nodes_used_collection_shard2_replica2/] nUpdates=100
[junit4:junit4]   2> 368612 T658 C9189 P59867 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4]   2> ASYNC  NEW_CORE C9190 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@1d3057 url=http://127.0.0.1:59867/nodes_used_collection_shard2_replica1 node=127.0.0.1:59867_ C9190_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:59867_, base_url=http://127.0.0.1:59867}
[junit4:junit4]   2> 368626 T658 C9190 P59867 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 368626 T658 C9190 P59867 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 368627 T658 C9190 P59867 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 368627 T658 C9190 P59867 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 368627 T658 C9190 P59867 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:59858/nodes_used_collection_shard2_replica2/. core=nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 368627 T658 C9190 P59867 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C9191 name=nodes_used_collection_shard1_replica1 org.apache.solr.core.SolrCore@1984e73 url=http://127.0.0.1:59828/nodes_used_collection_shard1_replica1 node=127.0.0.1:59828_ C9191_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica1 props:{shard=shard1, state=recovering, core=nodes_used_collection_shard1_replica1, collection=nodes_used_collection, node_name=127.0.0.1:59828_, base_url=http://127.0.0.1:59828}
[junit4:junit4]   2> 368635 T659 C9191 P59828 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:59849/nodes_used_collection_shard1_replica2/ core=nodes_used_collection_shard1_replica1 - recoveringAfterStartup=true
[junit4:junit4]   2> 368635 T659 C9191 P59828 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard1_replica1 url=http://127.0.0.1:59828 START replicas=[http://127.0.0.1:59849/nodes_used_collection_shard1_replica2/] nUpdates=100
[junit4:junit4]   2> 368637 T659 C9191 P59828 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 368640 T659 C9191 P59828 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=nodes_used_collection_shard1_replica1
[junit4:junit4]   2> ASYNC  NEW_CORE C9192 name=nodes_used_collection_shard1_replica1 org.apache.solr.core.SolrCore@1984e73 url=http://127.0.0.1:59828/nodes_used_collection_shard1_replica1 node=127.0.0.1:59828_ C9192_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica1 props:{shard=shard1, state=recovering, core=nodes_used_collection_shard1_replica1, collection=nodes_used_collection, node_name=127.0.0.1:59828_, base_url=http://127.0.0.1:59828}
[junit4:junit4]   2> 368667 T659 C9192 P59828 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=nodes_used_collection_shard1_replica1
[junit4:junit4]   2> 368667 T659 C9192 P59828 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=nodes_used_collection_shard1_replica1
[junit4:junit4]   2> 368667 T659 C9192 P59828 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 368667 T659 C9192 P59828 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:59849/nodes_used_collection_shard1_replica2/. core=nodes_used_collection_shard1_replica1
[junit4:junit4]   2> 368667 T659 C9192 P59828 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 368802 T596 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 368856 T628 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 368859 T596 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 368920 T628 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 369003 T596 oass.IndexSchema.readSchema [awholynewcollection_0_shard1_replica2] Schema name=test
[junit4:junit4]   2> 369044 T628 oass.IndexSchema.readSchema [awholynewcollection_0_shard4_replica3] Schema name=test
[junit4:junit4]   2> ASYNC  NEW_CORE C9193 name=nodes_used_collection_shard2_replica2 org.apache.solr.core.SolrCore@fef39d url=http://127.0.0.1:59858/nodes_used_collection_shard2_replica2 node=127.0.0.1:59858_ C9193_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica2 props:{shard=shard2, state=active, core=nodes_used_collection_shard2_replica2, collection=nodes_used_collection, node_name=127.0.0.1:59858_, base_url=http://127.0.0.1:59858, leader=true}
[junit4:junit4]   2> 369208 T626 C9193 P59858 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> ASYNC  NEW_CORE C9194 name=nodes_used_collection_shard2_replica2 org.apache.solr.core.SolrCore@fef39d url=http://127.0.0.1:59858/nodes_used_collection_shard2_replica2 node=127.0.0.1:59858_ C9194_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica2 props:{shard=shard2, state=active, core=nodes_used_collection_shard2_replica2, collection=nodes_used_collection, node_name=127.0.0.1:59858_, base_url=http://127.0.0.1:59858, leader=true}
[junit4:junit4]   2> 369248 T626 C9194 P59858 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1120709 lockFactory=org.apache.lucene.store.NativeFSLockFactory@f20092),segFN=segments_1,generation=1}
[junit4:junit4]   2> ASYNC  NEW_CORE C9195 name=nodes_used_collection_shard2_replica2 org.apache.solr.core.SolrCore@fef39d url=http://127.0.0.1:59858/nodes_used_collection_shard2_replica2 node=127.0.0.1:59858_ C9195_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica2 props:{shard=shard2, state=active, core=nodes_used_collection_shard2_replica2, collection=nodes_used_collection, node_name=127.0.0.1:59858_, base_url=http://127.0.0.1:59858, leader=true}
[junit4:junit4]   2> 369279 T626 C9195 P59858 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 369294 T626 C9195 P59858 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1120709 lockFactory=org.apache.lucene.store.NativeFSLockFactory@f20092),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1120709 lockFactory=org.apache.lucene.store.NativeFSLockFactory@f20092),segFN=segments_2,generation=2}
[junit4:junit4]   2> 369294 T626 C9195 P59858 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 369295 T626 C9195 P59858 oass.SolrIndexSearcher.<init> Opening Searcher@1113b1a realtime
[junit4:junit4]   2> ASYNC  NEW_CORE C9196 name=nodes_used_collection_shard2_replica2 org.apache.solr.core.SolrCore@fef39d url=http://127.0.0.1:59858/nodes_used_collection_shard2_replica2 node=127.0.0.1:59858_ C9196_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica2 props:{shard=shard2, state=active, core=nodes_used_collection_shard2_replica2, collection=nodes_used_collection, node_name=127.0.0.1:59858_, base_url=http://127.0.0.1:59858, leader=true}
[junit4:junit4]   2> 369306 T626 C9196 P59858 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 369306 T626 C9196 P59858 oasup.LogUpdateProcessor.finish [nodes_used_collection_shard2_replica2] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 99
[junit4:junit4]   2> ASYNC  NEW_CORE C9197 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@1d3057 url=http://127.0.0.1:59867/nodes_used_collection_shard2_replica1 node=127.0.0.1:59867_ C9197_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:59867_, base_url=http://127.0.0.1:59867}
[junit4:junit4]   2> 369308 T658 C9197 P59867 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 369309 T658 C9197 P59867 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 369313 T690 C9196 P59858 oasc.SolrCore.execute [nodes_used_collection_shard2_replica2] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1 
[junit4:junit4]   2> ASYNC  NEW_CORE C9198 name=nodes_used_collection_shard2_replica2 org.apache.solr.core.SolrCore@fef39d url=http://127.0.0.1:59858/nodes_used_collection_shard2_replica2 node=127.0.0.1:59858_ C9198_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica2 props:{shard=shard2, state=active, core=nodes_used_collection_shard2_replica2, collection=nodes_used_collection, node_name=127.0.0.1:59858_, base_url=http://127.0.0.1:59858, leader=true}
[junit4:junit4]   2> 369339 T690 C9198 P59858 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 369339 T690 C9198 P59858 oasc.SolrCore.execute [nodes_used_collection_shard2_replica2] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> ASYNC  NEW_CORE C9199 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@1d3057 url=http://127.0.0.1:59867/nodes_used_collection_shard2_replica1 node=127.0.0.1:59867_ C9199_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:59867_, base_url=http://127.0.0.1:59867}
[junit4:junit4]   2> 369343 T658 C9199 P59867 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 369345 T658 C9199 P59867 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 369345 T658 C9199 P59867 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> ASYNC  NEW_CORE C9200 name=nodes_used_collection_shard2_replica2 org.apache.solr.core.SolrCore@fef39d url=http://127.0.0.1:59858/nodes_used_collection_shard2_replica2 node=127.0.0.1:59858_ C9200_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica2 props:{shard=shard2, state=active, core=nodes_used_collection_shard2_replica2, collection=nodes_used_collection, node_name=127.0.0.1:59858_, base_url=http://127.0.0.1:59858, leader=true}
[junit4:junit4]   2> 369381 T626 C9200 P59858 oasc.SolrCore.execute [nodes_used_collection_shard2_replica2] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=10 
[junit4:junit4]   2> ASYNC  NEW_CORE C9201 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@1d3057 url=http://127.0.0.1:59867/nodes_used_collection_shard2_replica1 node=127.0.0.1:59867_ C9201_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:59867_, base_url=http://127.0.0.1:59867}
[junit4:junit4]   2> 369382 T658 C9201 P59867 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 369382 T579 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 369385 T658 C9201 P59867 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226\nodes_used_collection_shard2_replica1\data\index.20130605012003443
[junit4:junit4]   2> ASYNC  NEW_CORE C9202 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@1d3057 url=http://127.0.0.1:59867/nodes_used_collection_shard2_replica1 node=127.0.0.1:59867_ C9202_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:59867_, base_url=http://127.0.0.1:59867}
[junit4:junit4]   2> 369401 T658 C9202 P59867 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@9a97b5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1db7ed8) fullCopy=false
[junit4:junit4]   2> 369491 T579 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> ASYNC  NEW_CORE C9203 name=nodes_used_collection_shard2_replica2 org.apache.solr.core.SolrCore@fef39d url=http://127.0.0.1:59858/nodes_used_collection_shard2_replica2 node=127.0.0.1:59858_ C9203_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica2 props:{shard=shard2, state=active, core=nodes_used_collection_shard2_replica2, collection=nodes_used_collection, node_name=127.0.0.1:59858_, base_url=http://127.0.0.1:59858, leader=true}
[junit4:junit4]   2> 369494 T690 C9203 P59858 oasc.SolrCore.execute [nodes_used_collection_shard2_replica2] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> ASYNC  NEW_CORE C9204 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@1d3057 url=http://127.0.0.1:59867/nodes_used_collection_shard2_replica1 node=127.0.0.1:59867_ C9204_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:59867_, base_url=http://127.0.0.1:59867}
[junit4:junit4]   2> 369495 T658 C9204 P59867 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> ASYNC  NEW_CORE C9205 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@1d3057 url=http://127.0.0.1:59867/nodes_used_collection_shard2_replica1 node=127.0.0.1:59867_ C9205_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:59867_, base_url=http://127.0.0.1:59867}
[junit4:junit4]   2> 369523 T658 C9205 P59867 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 369524 T658 C9205 P59867 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 369531 T658 C9205 P59867 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1891974 lockFactory=org.apache.lucene.store.NativeFSLockFactory@10ad99c),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1891974 lockFactory=org.apache.lucene.store.NativeFSLockFactory@10ad99c),segFN=segments_2,generation=2}
[junit4:junit4]   2> 369550 T658 C9205 P59867 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> ASYNC  NEW_CORE C9206 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@1d3057 url=http://127.0.0.1:59867/nodes_used_collection_shard2_replica1 node=127.0.0.1:59867_ C9206_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:59867_, base_url=http://127.0.0.1:59867}
[junit4:junit4]   2> 369673 T658 C9206 P59867 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 369673 T658 C9206 P59867 oass.SolrIndexSearcher.<init> Opening Searcher@1139ebd main
[junit4:junit4]   2> 369676 T653 oasc.SolrCore.registerSearcher [nodes_used_collection_shard2_replica1] Registered new searcher Searcher@1139ebd main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 369679 T658 C9206 P59867 oasc.CachingDirectoryFactory.closeCacheValue looking to close C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226\nodes_used_collection_shard2_replica1\data\index.20130605012003443 [CachedDir<<refCount=0;path=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226\nodes_used_collection_shard2_replica1\data\index.20130605012003443;done=true>>]
[junit4:junit4]   2> 369679 T658 C9206 P59867 oasc.CachingDirectoryFactory.close Closing directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226\nodes_used_collection_shard2_replica1\data\index.20130605012003443
[junit4:junit4]   2> 369679 T658 C9206 P59867 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370409585226\nodes_used_collection_shard2_replica1\data\index.20130605012003443
[junit4:junit4]   2> 369680 T658 C9206 P59867 oasc.RecoveryStrategy.replay No replay needed. core=nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 369680 T658 C9206 P59867 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 369680 T658 C9206 P59867 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica1 state=active
[junit4:junit4]   2> ASYNC  NEW_CORE C9207 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@1d3057 url=http://127.0.0.1:59867/nodes_used_collection_shard2_replica1 node=127.0.0.1:59867_ C9207_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:59867_, base_url=http://127.0.0.1:59867}
[junit4:junit4]   2> 369730 T658 C9207 P59867 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=nodes_used_collection_shard2_replica1
[junit4:junit4]   2> ASYNC  NEW_CORE C9208 name=nodes_used_collection_shard1_replica2 org.apache.solr.core.SolrCore@1971d19 url=http://127.0.0.1:59849/nodes_used_collection_shard1_replica2 node=127.0.0.1:59849_ C9208_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica2 props:{shard=shard1, state=active, core=nodes_used_collection_shard1_replica2, collection=nodes_used_collection, node_name=127.0.0.1:59849_, base_url=http://127.0.0.1:59849, leader=true}
[junit4:junit4]   2> 369920 T689 C9208 P59849 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> ASYNC  NEW_CORE C9209 name=nodes_used_collection_shard1_replica2 org.apache.solr.core.SolrCore@1971d19 url=http://127.0.0.1:59849/nodes_used_collection_shard1_replica2 node=127.0.0.1:59849_ C9209_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica2 props:{shard=shard1, state=active, core=nodes_used_collection_shard1_replica2, collection=nodes_used_collection, node_name=127.0.0.1:59849_, base_url=http://127.0.0.1:59849, leader=true}
[junit4:junit4]   2> 370021 T689 C9209 P59849 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1732a6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@83f397),segFN=segments_1,generation=1}
[junit4:junit4]   2> ASYNC  NEW_CORE C9210 name=nodes_used_collection_shard1_replica2 org.apache.solr.core.SolrCore@1971d19 url=http://127.0.0.1:59849/nodes_used_collection_shard1_replica2 node=127.0.0.1:59849_ C9210_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica2 props:{shard=shard1, state=active, core=nodes_used_collection_shard1_replica2, collection=nodes_used_collection, node_name=127.0.0.1:59849_, base_url=http://127.0.0.1:59849, leader=true}
[junit4:junit4]   2> 370049 T689 C9210 P59849 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 370050 T689 C9210 P59849 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1732a6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@83f397),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1732a6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@83f397),segFN=segments_2,generation=2}
[junit4:junit4]   2> 370050 T689 C9210 P59849 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 370051 T689 C9210 P59849 oass.SolrIndexSearcher.<init> Opening Searcher@12eb8ed realtime
[junit4:junit4]   2> 370051 T689 C9210 P59849 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 370051 T689 C9210 P59849 oasup.LogUpdateProcessor.finish [nodes_used_collection_shard1_replica2] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 131
[junit4:junit4]   2> ASYNC  NEW_CORE C9211 name=nodes_used_collection_shard1_replica1 org.apache.solr.core.SolrCore@1984e73 url=http://127.0.0.1:59828/nodes_used_collection_shard1_replica1 node=127.0.0.1:59828_ C9211_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica1 props:{shard=shard1, state=recovering, core=nodes_used_collection_shard1_replica1, collection=nodes_used_collection, node_name=127.0.0.1:59828_, base_url=http://127.0.0.1:59828}
[junit4:junit4]   2> 370054 T659 C9211 P59828 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 370054 T659 C9211 P59828 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> ASYNC  NEW_CORE C9212 name=nodes_used_collection_shard1_replica2 org.apache.solr.core.SolrCore@1971d19 url=http://127.0.0.1:59849/nodes_used_collection_shard1_replica2 node=127.0.0.1:59849_ C9212_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica2 props:{shard=shard1, state=active, core=nodes_used_collection_shard1_replica2, collection=nodes_used_collection, node_name=127.0.0.1:59849_, base_url=http://127.0.0.1:59849, leader=true}
[junit4:junit4]   2> 370123 T689 C9212 P59849 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 370125 T689 C9212 P59849 oasc.SolrCore.execute [nodes_used_collection_shard1_replica2] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> ASYNC  NEW_CORE C9213 name=nodes_used_collection_shard1_replica1 org.apache.solr.core.SolrCore@1984e73 url=http://127.0.0.1:59828/nodes_used_collection_shard1_replica1 node=127.0.0.1:59828_ C9213_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica1 props:{shard=shard1, state=recovering, core=nodes_used_collection_shard1_replica1, collection=nodes_used_collection, node_name=127.0.0.1:59828_, base_url=http://127.0.0.1:59828}
[junit4:junit4]   2> 370125 T659 C9213 P59828 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 370125 T659 C9213 P59828 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 370125 T659 C9213 P59828 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 370192 T579 oass.IndexSchema.readSchema [awholynewcollection_0_shard3_replica5] Schema name=test
[junit4:junit4]   2> ASYNC  NEW_CORE C9214 name=nodes_used_collection_shard1_replica2 org.apache.solr.core.SolrCore@1971d19 url=http://127.0.0.1:59849/nodes_used_collection_shard1_replica2 node=127.0.0.1:59849_ C9214_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica2 props:{shard=shard1, state=active, core=nodes_used_collection_shard1_replica2, collection=nodes_used_collection, node_name=127.0.0.1:59849_, base_url=http://127.0.0.1:59849, leader=true}
[junit4:junit4]   2> 370218 T694 C9214 P59849 oasc.SolrCore.execute [nodes_used_collection_shard1_replica2] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0 
[junit4:junit4]   2> ASYNC  NEW_CORE C9215 name=nodes_used_collection_shard1_replica1 org.apache.solr.core.SolrCore@1984e73 url=http://127.0.0.1:59828/nodes_used_collection_shard1_replica1 node=127.0.0.1:59828_ C9215_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica1 props:{shard=shard1, state=recovering, core=nodes_used_collection_shard1_replica1, collection=nodes_used_collection, node_name=127.0.0.1:59828_, base_url=http://127.0.0.1:59828}
[junit4:junit4]   2> 370220 T659 C9215 P59828 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> ASYNC  NEW_CORE C9216 name=nodes_used_collection_shard1_replica1 org.apache.solr.core.SolrCore@1984e73 url=http://127.0.0.1:59828/nodes_used_collection_shard1_replica1 node=127.0.0.1:59828_ C9216_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica1 props:{shard=shard1, state=recovering, core=nodes_used_collection_shard1_replica1, collection=nodes_used_collection, node_name=127.0.0.1:59828_, base_url=http://127.0.0.1:59828}
[junit4:junit4]   2> 370245 T659 C9216 P59828 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370409576781\nodes_used_collection_shard1_replica1\data\index.20130605012004282
[junit4:junit4]   2> 370246 T659 C9216 P59828 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@1a0e4ec lockFactory=org.apache.lucene.store.NativeFSLockFactory@c6129c) fullCopy=false
[junit4:junit4]   2> ASYNC  NEW_CORE C9217 name=nodes_used_collection_shard1_replica2 org.apache.solr.core.SolrCore@1971d19 url=http://127.0.0.1:59849/nodes_used_collection_shard1_replica2 node=127.0.0.1:59849_ C9217_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica2 props:{shard=shard1, state=active, core=nodes_used_collection_shard1_replica2, collection=nodes_used_collection, node_name=127.0.0.1:59849_, base_url=http://127.0.0.1:59849, leader=true}
[junit4:junit4]   2> 370268 T689 C9217 P59849 oasc.SolrCore.execute [nodes_used_collection_shar

[...truncated too long message...]

.ZkTestServer.send4LetterWord connecting to 127.0.0.1:63093 63093
[junit4:junit4]   2> 2670190 T6995 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 2671259 T6995 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 2671262 T6995 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 2671262 T6904 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 63100
[junit4:junit4]   2> 2671279 T6904 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4]   2> 2671280 T6904 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:63093 63093
[junit4:junit4]   1> client port:0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ShardSplitTest -Dtests.method=testDistribSearch -Dtests.seed=E4C1C47DC3F0BC1C -Dtests.slow=true -Dtests.locale=es_PY -Dtests.timezone=Etc/GMT-14 -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR   66.4s | ShardSplitTest.testDistribSearch <<<
[junit4:junit4]    > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:63112 returned non ok status:500, message:Server Error
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([E4C1C47DC3F0BC1C:65274A65B4AFDC20]:0)
[junit4:junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
[junit4:junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4]    > 	at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
[junit4:junit4]    > 	at org.apache.solr.cloud.ShardSplitTest.doTest(ShardSplitTest.java:150)
[junit4:junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 2671286 T6904 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> NOTE: test params are: codec=Lucene42: {id=Pulsing41(freqCutoff=19 minBlockSize=4 maxBlockSize=101), _version_=PostingsFormat(name=TestBloomFilteredLucene41Postings), n_ti=Lucene41(blocksize=128)}, docValues:{}, sim=DefaultSimilarity, locale=es_PY, timezone=Etc/GMT-14
[junit4:junit4]   2> NOTE: Windows 7 6.1 x86/Oracle Corporation 1.7.0_21 (32-bit)/cpus=2,threads=3,free=214217952,total=329490432
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestTrie, AlternateDirectoryTest, ScriptEngineTest, TestSolrCoreProperties, TestStressRecovery, TestCopyFieldCollectionResource, TestNumberUtils, SolrCoreCheckLockOnStartupTest, TestPseudoReturnFields, TestDocumentBuilder, TestWordDelimiterFilterFactory, SpellPossibilityIteratorTest, SolrIndexConfigTest, TestDefaultSearchFieldResource, SuggesterTest, TestMergePolicyConfig, ChaosMonkeySafeLeaderTest, TestAddFieldRealTimeGet, ResourceLoaderTest, TestZkChroot, DOMUtilTest, UpdateRequestProcessorFactoryTest, TestFaceting, CollectionsAPIDistributedZkTest, TestFunctionQuery, SolrTestCaseJ4Test, TestRTGBase, TestSolrXMLSerializer, CopyFieldTest, TestDynamicFieldCollectionResource, SliceStateTest, IndexSchemaTest, TestRandomDVFaceting, SolrCoreTest, SolrInfoMBeanTest, TestSearchPerf, PolyFieldTest, TestJmxIntegration, SpatialFilterTest, TestUniqueKeyFieldResource, TestFoldingMultitermQuery, QueryParsingTest, TestBM25SimilarityFactory, PrimUtilsTest, TestSolrJ, TestCharFilters, RAMDirectoryFactoryTest, UpdateParamsTest, SoftAutoCommitTest, TestDocSet, TestLMJelinekMercerSimilarityFactory, TestSweetSpotSimilarityFactory, SOLR749Test, TestUpdate, TermVectorComponentTest, TimeZoneUtilsTest, TestMultiCoreConfBootstrap, TestValueSourceCache, OverseerTest, TestComponentsName, PluginInfoTest, TestPerFieldSimilarity, TestJmxMonitoredMap, TestSchemaSimilarityResource, TestLuceneMatchVersion, SpellCheckComponentTest, TestAnalyzedSuggestions, TestDefaultSimilarityFactory, TestLFUCache, NumericFieldsTest, MinimalSchemaTest, DocumentBuilderTest, TestSerializedLuceneMatchVersion, MBeansHandlerTest, TestReversedWildcardFilterFactory, TestHashPartitioner, TestCoreDiscovery, OpenCloseCoreStressTest, FastVectorHighlighterTest, TestConfig, SimplePostToolTest, SearchHandlerTest, TestBinaryField, TestReplicationHandler, IndexReaderFactoryTest, TestFastOutputStream, TestSchemaVersionResource, TestDFRSimilarityFactory, JsonLoaderTest, RecoveryZkTest, PreAnalyzedFieldTest, CoreContainerCoreInitFailuresTest, CurrencyFieldXmlFileTest, TestPostingsSolrHighlighter, ZkControllerTest, SolrRequestParserTest, TestSchemaNameResource, JSONWriterTest, BadIndexSchemaTest, NotRequiredUniqueKeyTest, TestSolrDeletionPolicy1, DefaultValueUpdateProcessorTest, RequiredFieldsTest, LeaderElectionTest, TestPhraseSuggestions, TestSolrQueryParser, TestSuggestSpellingConverter, TestReload, AutoCommitTest, MultiTermTest, TestFieldTypeCollectionResource, BadComponentTest, TestQuerySenderNoQuery, TestQueryTypes, BinaryUpdateRequestHandlerTest, UniqFieldsUpdateProcessorFactoryTest, DistributedTermsComponentTest, BasicDistributedZkTest, DistributedSpellCheckComponentTest, SolrCmdDistributorTest, SampleTest, TestCloudManagedSchema, HighlighterTest, CacheHeaderTest, TestCodecSupport, SynonymTokenizerTest, TestManagedSchema, ZkCLITest, TestRemoteStreaming, CircularListTest, TestFastWriter, HighlighterConfigTest, DistributedQueryElevationComponentTest, ShowFileRequestHandlerTest, TestStressReorder, TestFieldTypeResource, TestSolrIndexConfig, TestIndexingPerformance, IndexBasedSpellCheckerTest, ShardRoutingTest, BasicDistributedZk2Test, UnloadDistributedZkTest, SyncSliceTest, LeaderElectionIntegrationTest, BasicZkTest, FullSolrCloudDistribCmdsTest, ClusterStateUpdateTest, TestRandomFaceting, ZkSolrClientTest, ShardRoutingCustomTest, SliceStateUpdateTest, TestDistributedSearch, TestDistributedGrouping, TestRecovery, TermVectorComponentDistributedTest, TestRealTimeGet, TestJoin, TestStressVersions, HardAutoCommitTest, TestRangeQuery, TestCoreContainer, SimpleFacetsTest, TestSolr4Spatial, StatsComponentTest, TestGroupingSearch, QueryElevationComponentTest, PeerSyncTest, ConvertedLegacyTest, TestSort, TestFiltering, BasicFunctionalityTest, TestLazyCores, DirectUpdateHandlerTest, TestBadConfig, OverseerCollectionProcessorTest, TestIndexSearcher, CurrencyFieldOpenExchangeTest, SolrIndexSplitterTest, AnalysisAfterCoreReloadTest, SignatureUpdateProcessorFactoryTest, TestExtendedDismaxParser, SpellCheckCollatorTest, SuggesterFSTTest, CoreAdminHandlerTest, DocValuesTest, SuggesterTSTTest, TestStressLucene, SuggesterWFSTTest, TestCSVLoader, NoCacheHeaderTest, WordBreakSolrSpellCheckerTest, SchemaVersionSpecificBehaviorTest, FieldMutatingUpdateProcessorTest, TestAtomicUpdateErrorCases, QueryEqualityTest, DirectUpdateHandlerOptimizeTest, StatelessScriptUpdateProcessorFactoryTest, SortByFunctionTest, DocValuesMultiTest, DistanceFunctionTest, XsltUpdateRequestHandlerTest, DebugComponentTest, TestSurroundQueryParser, LukeRequestHandlerTest, DisMaxRequestHandlerTest, TestQueryUtils, StandardRequestHandlerTest, TestWriterPerf, DirectSolrSpellCheckerTest, DocumentAnalysisRequestHandlerTest, PrimitiveFieldTypeTest, TestOmitPositions, FileBasedSpellCheckerTest, TermsComponentTest, XmlUpdateRequestHandlerTest, RequestHandlersTest, PathHierarchyTokenizerFactoryTest, MoreLikeThisHandlerTest, FieldAnalysisRequestHandlerTest, TestArbitraryIndexDir, IndexSchemaRuntimeFieldTest, LoggingHandlerTest, RegexBoostProcessorTest, SolrPluginUtilsTest, TestCollationField, ReturnFieldsTest, TestCSVResponseWriter, TestPartialUpdateDeduplication, TestPHPSerializedResponseWriter, PingRequestHandlerTest, CSVRequestHandlerTest, TestBinaryResponseWriter, TestPropInject, TestQuerySenderListener, TestPropInjectDefaults, ResponseLogComponentTest, TestSolrDeletionPolicy2, TestElisionMultitermQuery, OutputWriterTest, TestFuzzyAnalyzedSuggestions, ExternalFileFieldSortTest, DirectSolrConnectionTest, TestXIncludeConfig, EchoParamsTest, TestLMDirichletSimilarityFactory, TestIBSimilarityFactory, LegacyHTMLStripCharFilterTest, TestPluginEnable, OpenExchangeRatesOrgProviderTest, URLClassifyProcessorTest, TestFastLRUCache, DateMathParserTest, ChaosMonkeyNothingIsSafeTest, TestSystemIdResolver, DateFieldTest, SpellingQueryConverterTest, ClusterStateTest, TestLRUCache, TestUtils, ZkNodePropsTest, SystemInfoHandlerTest, UUIDFieldTest, FileUtilsTest, AliasIntegrationTest, ChaosMonkeyShardSplitTest, ShardSplitTest]
[junit4:junit4] Completed in 66.43s, 1 test, 1 error <<< FAILURES!

[...truncated 62 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:380: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:360: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:39: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build.xml:181: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\common-build.xml:437: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1240: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:884: There were test failures: 296 suites, 1235 tests, 3 errors, 18 ignored (12 assumptions)

Total time: 69 minutes 59 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.7.0_21 -client -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-trunk-Windows (32bit/jdk1.7.0_21) - Build # 2891 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/2891/
Java: 32bit/jdk1.7.0_21 -server -XX:+UseConcMarkSweepGC

1 tests failed.
REGRESSION:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch

Error Message:
There are still nodes recoverying - waited for 230 seconds

Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 230 seconds
	at __randomizedtesting.SeedInfo.seed([28BD50ED2F50C07D:A95BDEF5580FA041]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:173)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:126)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:512)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:146)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)




Build Log:
[...truncated 9561 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
[junit4:junit4]   2> 1556708 T3794 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4]   2> 1556709 T3794 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\solrtest-CollectionsAPIDistributedZkTest-1370387428340
[junit4:junit4]   2> 1556712 T3794 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 1556714 T3795 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 1556802 T3794 oasc.ZkTestServer.run start zk server on port:53424
[junit4:junit4]   2> 1556804 T3794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1556812 T3801 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a6e8dd name:ZooKeeperConnection Watcher:127.0.0.1:53424 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1556812 T3794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1556812 T3794 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 1556821 T3796 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13f1174b3fe0000, likely client has closed socket
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 
[junit4:junit4]   2> 1556823 T3794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1556827 T3803 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7a9510 name:ZooKeeperConnection Watcher:127.0.0.1:53424/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1556827 T3794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1556827 T3794 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 1556833 T3794 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 1556837 T3794 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 1556842 T3794 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 1556846 T3794 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 1556847 T3794 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 1556856 T3794 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 1556857 T3794 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 1556862 T3794 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 1556863 T3794 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 1556868 T3794 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 1556868 T3794 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 1556873 T3794 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 1556874 T3794 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 1556878 T3794 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 1556879 T3794 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 1556883 T3794 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 1556884 T3794 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 1556888 T3794 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 1556889 T3794 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 1556893 T3794 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 1556894 T3794 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 1557382 T3794 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 1557387 T3794 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:53431
[junit4:junit4]   2> 1557388 T3794 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 1557389 T3794 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1557389 T3794 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535
[junit4:junit4]   2> 1557390 T3794 oasc.CoreContainer$Initializer.initialize looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535\solr.xml
[junit4:junit4]   2> 1557390 T3794 oasc.CoreContainer.<init> New CoreContainer 5098034
[junit4:junit4]   2> 1557390 T3794 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535\'
[junit4:junit4]   2> 1557391 T3794 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535\'
[junit4:junit4]   2> 1557490 T3794 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535
[junit4:junit4]   2> 1557492 T3794 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535\collection1
[junit4:junit4]   2> 1557494 T3794 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535\collection1\conf
[junit4:junit4]   2> 1557727 T3794 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535\collection1\conf\xslt
[junit4:junit4]   2> 1557735 T3794 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535\collection1\lib
[junit4:junit4]   2> 1557738 T3794 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535\collection1\lib\classes
[junit4:junit4]   2> 1557745 T3794 oasc.SolrCoreDiscoverer.addCore Discovered properties file C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535\conf\core.properties, adding to cores
[junit4:junit4]   2> 1557762 T3794 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 1557763 T3794 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 1557763 T3794 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 1557764 T3794 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 1557764 T3794 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 1557764 T3794 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 1557765 T3794 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 1557765 T3794 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 1557765 T3794 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 1557765 T3794 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 1557773 T3794 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 1557773 T3794 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:53424/solr
[junit4:junit4]   2> 1557773 T3794 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 1557776 T3794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1557785 T3814 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d7a43d name:ZooKeeperConnection Watcher:127.0.0.1:53424 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1557785 T3794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1557793 T3794 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 1557802 T3794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1557815 T3816 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6fc73b name:ZooKeeperConnection Watcher:127.0.0.1:53424/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1557815 T3794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1557821 T3794 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4]   2> 1557831 T3794 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4]   2> 1557839 T3794 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 1557842 T3794 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53431_
[junit4:junit4]   2> 1557845 T3794 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:53431_
[junit4:junit4]   2> 1557853 T3794 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 1557867 T3794 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 1557872 T3794 oasc.Overseer.start Overseer (id=89809710504345603-127.0.0.1:53431_-n_0000000000) starting
[junit4:junit4]   2> 1557880 T3794 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4]   2> 1557908 T3818 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 1557909 T3794 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 1557916 T3794 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4]   2> 1557920 T3794 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1557925 T3817 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 1557928 T3794 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
[junit4:junit4]   2> 1557929 T3794 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 1557929 T3794 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 1557932 T3794 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 1557935 T3794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1557938 T3821 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d0d1cc name:ZooKeeperConnection Watcher:127.0.0.1:53424/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1557939 T3794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1557941 T3794 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1557946 T3794 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 1558415 T3794 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 1558420 T3794 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:53443
[junit4:junit4]   2> 1558422 T3794 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 1558423 T3794 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1558423 T3794 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370387429575
[junit4:junit4]   2> 1558424 T3794 oasc.CoreContainer$Initializer.initialize looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370387429575\solr.xml
[junit4:junit4]   2> 1558424 T3794 oasc.CoreContainer.<init> New CoreContainer 24657411
[junit4:junit4]   2> 1558425 T3794 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370387429575\'
[junit4:junit4]   2> 1558425 T3794 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370387429575\'
[junit4:junit4]   2> 1558518 T3794 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370387429575
[junit4:junit4]   2> 1558520 T3794 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370387429575\collection1
[junit4:junit4]   2> 1558521 T3794 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370387429575\collection1\conf
[junit4:junit4]   2> 1558861 T3794 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370387429575\collection1\conf\xslt
[junit4:junit4]   2> 1558874 T3794 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370387429575\collection1\lib
[junit4:junit4]   2> 1558879 T3794 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370387429575\collection1\lib\classes
[junit4:junit4]   2> 1558889 T3794 oasc.SolrCoreDiscoverer.addCore Discovered properties file C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370387429575\conf\core.properties, adding to cores
[junit4:junit4]   2> 1558918 T3794 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 1558918 T3794 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 1558919 T3794 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 1558919 T3794 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 1558920 T3794 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 1558921 T3794 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 1558921 T3794 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 1558922 T3794 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 1558922 T3794 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 1558923 T3794 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 1558934 T3794 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 1558935 T3794 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:53424/solr
[junit4:junit4]   2> 1558935 T3794 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 1558937 T3794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1558951 T3832 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@18bc86e name:ZooKeeperConnection Watcher:127.0.0.1:53424 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1558953 T3794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1558957 T3794 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 1558957 T3796 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13f1174b3fe0005, likely client has closed socket
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 
[junit4:junit4]   2> 1558967 T3794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1558971 T3834 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e05574 name:ZooKeeperConnection Watcher:127.0.0.1:53424/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1558972 T3794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1558982 T3794 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1559865 T3794 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53443_
[junit4:junit4]   2> 1559868 T3794 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:53443_
[junit4:junit4]   2> 1559874 T3816 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 1559875 T3834 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 1559875 T3821 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 1559884 T3794 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
[junit4:junit4]   2> 1559884 T3794 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 1559884 T3794 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 1560335 T3794 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 1560342 T3794 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:53452
[junit4:junit4]   2> 1560342 T3794 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 1560343 T3794 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1560343 T3794 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370387431519
[junit4:junit4]   2> 1560343 T3794 oasc.CoreContainer$Initializer.initialize looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370387431519\solr.xml
[junit4:junit4]   2> 1560344 T3794 oasc.CoreContainer.<init> New CoreContainer 5062569
[junit4:junit4]   2> 1560344 T3794 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370387431519\'
[junit4:junit4]   2> 1560344 T3794 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370387431519\'
[junit4:junit4]   2> 1560430 T3794 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370387431519
[junit4:junit4]   2> 1560432 T3794 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370387431519\collection1
[junit4:junit4]   2> 1560433 T3794 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370387431519\collection1\conf
[junit4:junit4]   2> 1560677 T3794 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370387431519\collection1\conf\xslt
[junit4:junit4]   2> 1560684 T3794 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370387431519\collection1\lib
[junit4:junit4]   2> 1560686 T3794 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370387431519\collection1\lib\classes
[junit4:junit4]   2> 1560693 T3794 oasc.SolrCoreDiscoverer.addCore Discovered properties file C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370387431519\conf\core.properties, adding to cores
[junit4:junit4]   2> 1560708 T3794 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 1560708 T3794 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 1560709 T3794 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 1560709 T3794 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 1560709 T3794 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 1560710 T3794 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 1560710 T3794 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 1560710 T3794 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 1560711 T3794 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 1560711 T3794 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 1560717 T3794 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 1560718 T3794 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:53424/solr
[junit4:junit4]   2> 1560718 T3794 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 1560719 T3794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1560732 T3846 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@af9698 name:ZooKeeperConnection Watcher:127.0.0.1:53424 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1560732 T3794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1560736 T3794 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 1560736 T3796 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13f1174b3fe0007, likely client has closed socket
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 
[junit4:junit4]   2> 1560742 T3794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1560746 T3848 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c7a930 name:ZooKeeperConnection Watcher:127.0.0.1:53424/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1560746 T3794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1560753 T3794 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1561554 T3794 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53452_
[junit4:junit4]   2> 1561561 T3794 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:53452_
[junit4:junit4]   2> 1561566 T3834 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 1561566 T3848 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 1561566 T3821 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 1561567 T3816 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 1561573 T3794 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
[junit4:junit4]   2> 1561573 T3794 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 1561574 T3794 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 1561900 T3794 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 1561905 T3794 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:53461
[junit4:junit4]   2> 1561906 T3794 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 1561906 T3794 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1561906 T3794 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370387433206
[junit4:junit4]   2> 1561906 T3794 oasc.CoreContainer$Initializer.initialize looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370387433206\solr.xml
[junit4:junit4]   2> 1561906 T3794 oasc.CoreContainer.<init> New CoreContainer 16548676
[junit4:junit4]   2> 1561907 T3794 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370387433206\'
[junit4:junit4]   2> 1561907 T3794 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370387433206\'
[junit4:junit4]   2> 1561992 T3794 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370387433206
[junit4:junit4]   2> 1561994 T3794 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370387433206\collection1
[junit4:junit4]   2> 1561994 T3794 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370387433206\collection1\conf
[junit4:junit4]   2> 1562149 T3794 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370387433206\collection1\conf\xslt
[junit4:junit4]   2> 1562155 T3794 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370387433206\collection1\lib
[junit4:junit4]   2> 1562157 T3794 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370387433206\collection1\lib\classes
[junit4:junit4]   2> 1562162 T3794 oasc.SolrCoreDiscoverer.addCore Discovered properties file C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370387433206\conf\core.properties, adding to cores
[junit4:junit4]   2> 1562174 T3794 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 1562175 T3794 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 1562175 T3794 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 1562175 T3794 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 1562176 T3794 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 1562176 T3794 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 1562176 T3794 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 1562176 T3794 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 1562176 T3794 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 1562176 T3794 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 1562183 T3794 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 1562183 T3794 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:53424/solr
[junit4:junit4]   2> 1562183 T3794 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 1562187 T3794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1562202 T3860 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b8424c name:ZooKeeperConnection Watcher:127.0.0.1:53424 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1562203 T3794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1562206 T3794 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 1562206 T3796 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13f1174b3fe0009, likely client has closed socket
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 
[junit4:junit4]   2> 1562211 T3794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1562214 T3862 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9b1079 name:ZooKeeperConnection Watcher:127.0.0.1:53424/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1562215 T3794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1562222 T3794 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1563023 T3794 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53461_
[junit4:junit4]   2> 1563026 T3794 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:53461_
[junit4:junit4]   2> 1563031 T3821 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 1563031 T3834 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 1563031 T3862 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 1563032 T3848 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 1563032 T3816 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 1563040 T3794 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
[junit4:junit4]   2> 1563040 T3794 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 1563040 T3794 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 1563454 T3794 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 1563459 T3794 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:53470
[junit4:junit4]   2> 1563461 T3794 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 1563462 T3794 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1563462 T3794 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370387434673
[junit4:junit4]   2> 1563463 T3794 oasc.CoreContainer$Initializer.initialize looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370387434673\solr.xml
[junit4:junit4]   2> 1563463 T3794 oasc.CoreContainer.<init> New CoreContainer 18280810
[junit4:junit4]   2> 1563463 T3794 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370387434673\'
[junit4:junit4]   2> 1563464 T3794 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370387434673\'
[junit4:junit4]   2> 1563554 T3794 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370387434673
[junit4:junit4]   2> 1563556 T3794 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370387434673\collection1
[junit4:junit4]   2> 1563558 T3794 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370387434673\collection1\conf
[junit4:junit4]   2> 1563777 T3794 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370387434673\collection1\conf\xslt
[junit4:junit4]   2> 1563786 T3794 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370387434673\collection1\lib
[junit4:junit4]   2> 1563789 T3794 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370387434673\collection1\lib\classes
[junit4:junit4]   2> 1563796 T3794 oasc.SolrCoreDiscoverer.addCore Discovered properties file C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370387434673\conf\core.properties, adding to cores
[junit4:junit4]   2> 1563825 T3794 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 1563825 T3794 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 1563825 T3794 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 1563826 T3794 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 1563826 T3794 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 1563827 T3794 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 1563827 T3794 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 1563827 T3794 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 1563828 T3794 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 1563828 T3794 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 1563834 T3794 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 1563835 T3794 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:53424/solr
[junit4:junit4]   2> 1563835 T3794 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 1563839 T3794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1563850 T3874 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7104c5 name:ZooKeeperConnection Watcher:127.0.0.1:53424 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1563850 T3794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1563854 T3794 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 1563859 T3794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1563862 T3876 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1821b4c name:ZooKeeperConnection Watcher:127.0.0.1:53424/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1563863 T3794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1563870 T3794 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1564672 T3794 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53470_
[junit4:junit4]   2> 1564678 T3794 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:53470_
[junit4:junit4]   2> 1564685 T3821 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 1564686 T3862 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 1564686 T3834 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 1564686 T3876 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 1564686 T3816 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 1564686 T3848 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 1564694 T3794 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
[junit4:junit4]   2> 1564694 T3794 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 1564694 T3794 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 1564699 T3794 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 1564705 T3828 oasha.CollectionsHandler.handleCreateAction Creating Collection : numShards=2&name=nodes_used_collection&replicationFactor=2&action=CREATE&wt=javabin&version=2
[junit4:junit4]   2> 1564708 T3816 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4]   2> 1564712 T3818 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
[junit4:junit4]   2> 	  "operation":"createcollection",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "maxShardsPerNode":null,
[junit4:junit4]   2> 	  "createNodeSet":null,
[junit4:junit4]   2> 	  "name":"nodes_used_collection",
[junit4:junit4]   2> 	  "replicationFactor":"2"}
[junit4:junit4]   2> 1564712 T3818 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard1_replica1 as part of slice shard1 of collection nodes_used_collection on 127.0.0.1:53461_
[junit4:junit4]   2> 1564713 T3818 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard1_replica2 as part of slice shard1 of collection nodes_used_collection on 127.0.0.1:53470_
[junit4:junit4]   2> 1564714 T3818 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard2_replica1 as part of slice shard2 of collection nodes_used_collection on 127.0.0.1:53431_
[junit4:junit4]   2> 1564714 T3818 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard2_replica2 as part of slice shard2 of collection nodes_used_collection on 127.0.0.1:53452_
[junit4:junit4]   2> 1564717 T3857 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard1_replica1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370387433206\nodes_used_collection_shard1_replica1
[junit4:junit4]   2> 1564717 T3857 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4]   2> 1564717 T3812 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard2_replica1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535\nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 1564719 T3812 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4]   2> 1564718 T3868 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard1_replica2' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370387434673\nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 1564720 T3857 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4]   2> 1564721 T3857 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1564720 T3868 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4]   2> 1564719 T3838 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard2_replica2' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370387431519\nodes_used_collection_shard2_replica2
[junit4:junit4]   2> 1564722 T3838 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4]   2> 1564723 T3868 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4]   2> 1564723 T3868 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1564721 T3812 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4]   2> 1564724 T3812 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1564724 T3838 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4]   2> 1564724 T3838 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1564725 T3857 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 1564726 T3857 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4]   2> 1564727 T3868 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 1564727 T3868 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4]   2> 1564728 T3812 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 1564728 T3812 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4]   2> 1564730 T3838 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 1564731 T3838 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4]   2> 1564735 T3857 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4]   2> 1564736 T3868 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4]   2> 1564738 T3812 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4]   2> 1564739 T3838 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4]   2> 1564743 T3868 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370387434673\nodes_used_collection_shard1_replica2\'
[junit4:junit4]   2> 1564743 T3857 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370387433206\nodes_used_collection_shard1_replica1\'
[junit4:junit4]   2> 1564747 T3838 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370387431519\nodes_used_collection_shard2_replica2\'
[junit4:junit4]   2> 1564752 T3812 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535\nodes_used_collection_shard2_replica1\'
[junit4:junit4]   2> 1564830 T3812 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1564830 T3868 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1564880 T3857 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1564911 T3838 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1564941 T3857 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1564944 T3857 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1564949 T3812 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1564950 T3857 oass.IndexSchema.readSchema [nodes_used_collection_shard1_replica1] Schema name=test
[junit4:junit4]   2> 1564953 T3812 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1564956 T3812 oass.IndexSchema.readSchema [nodes_used_collection_shard2_replica1] Schema name=test
[junit4:junit4]   2> 1565015 T3838 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1565072 T3868 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1565187 T3838 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1565191 T3838 oass.IndexSchema.readSchema [nodes_used_collection_shard2_replica2] Schema name=test
[junit4:junit4]   2> 1565262 T3868 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1565266 T3868 oass.IndexSchema.readSchema [nodes_used_collection_shard1_replica2] Schema name=test
[junit4:junit4]   2> 1565628 T3857 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1565637 T3857 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1565641 T3857 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1566028 T3812 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1566038 T3812 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1566042 T3812 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1566057 T3857 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1566120 T3857 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1566127 T3857 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1566139 T3857 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1566144 T3857 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1566144 T3857 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1566215 T3857 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1566216 T3857 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1566216 T3857 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1566217 T3857 oasc.SolrCore.<init> [nodes_used_collection_shard1_replica1] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370387433206\nodes_used_collection_shard1_replica1\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370387433206\nodes_used_collection_shard1_replica1\data\
[junit4:junit4]   2> 1566217 T3857 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@ccb229
[junit4:junit4]   2> 1566218 T3857 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1566218 T3857 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370387433206\nodes_used_collection_shard1_replica1\data\
[junit4:junit4]   2> 1566220 T3857 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370387433206\nodes_used_collection_shard1_replica1\data\index/
[junit4:junit4]   2> 1566220 T3857 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard1_replica1] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370387433206\nodes_used_collection_shard1_replica1\data\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1566221 T3857 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370387433206\nodes_used_collection_shard1_replica1\data\index
[junit4:junit4]   2> 1566228 T3857 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1aea164 lockFactory=org.apache.lucene.store.NativeFSLockFactory@187bc5e),segFN=segments_1,generation=1}
[junit4:junit4]   2> 1566228 T3857 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 1566233 T3857 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1566234 T3857 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1566235 T3857 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1566236 T3857 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1566236 T3857 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1566236 T3857 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1566236 T3857 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1566238 T3857 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1566239 T3857 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1566276 T3857 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1566281 T3857 oass.SolrIndexSearcher.<init> Opening Searcher@a581ac main
[junit4:junit4]   2> 1566282 T3857 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370387433206\nodes_used_collection_shard1_replica1\data\tlog
[junit4:junit4]   2> 1566284 T3857 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1566285 T3857 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1566309 T3838 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1566320 T3838 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1566334 T3838 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1566342 T3857 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica1 state=down
[junit4:junit4]   2> 1566344 T3882 oasc.SolrCore.registerSearcher [nodes_used_collection_shard1_replica1] Registered new searcher Searcher@a581ac main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1566359 T3812 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1566382 T3812 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1566389 T3812 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1566390 T3838 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1566393 T3812 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1566394 T3812 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1566394 T3812 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1566399 T3838 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1566402 T3812 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1566403 T3812 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1566403 T3812 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1566404 T3812 oasc.SolrCore.<init> [nodes_used_collection_shard2_replica1] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535\nodes_used_collection_shard2_replica1\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535\nodes_used_collection_shard2_replica1\data\
[junit4:junit4]   2> 1566404 T3812 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@ccb229
[junit4:junit4]   2> 1566404 T3812 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1566404 T3812 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535\nodes_used_collection_shard2_replica1\data\
[junit4:junit4]   2> 1566409 T3838 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1566410 T3812 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535\nodes_used_collection_shard2_replica1\data\index/
[junit4:junit4]   2> 1566410 T3812 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard2_replica1] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535\nodes_used_collection_shard2_replica1\data\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1566411 T3812 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535\nodes_used_collection_shard2_replica1\data\index
[junit4:junit4]   2> 1566415 T3838 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1566415 T3838 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1566416 T3838 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1566418 T3812 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d2d28b lockFactory=org.apache.lucene.store.NativeFSLockFactory@4898a9),segFN=segments_1,generation=1}
[junit4:junit4]   2> 1566418 T3812 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 1566422 T3812 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1566422 T3812 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1566423 T3812 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1566424 T3812 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1566424 T3812 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1566425 T3812 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1566425 T3812 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1566426 T3812 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1566426 T3812 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1566436 T3838 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1566436 T3838 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1566436 T3838 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1566437 T3838 oasc.SolrCore.<init> [nodes_used_collection_shard2_replica2] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370387431519\nodes_used_collection_shard2_replica2\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370387431519\nodes_used_collection_shard2_replica2\data\
[junit4:junit4]   2> 1566437 T3838 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@ccb229
[junit4:junit4]   2> 1566438 T3838 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1566439 T3838 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370387431519\nodes_used_collection_shard2_replica2\data\
[junit4:junit4]   2> 1566444 T3868 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1566448 T3812 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1566451 T3838 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370387431519\nodes_used_collection_shard2_replica2\data\index/
[junit4:junit4]   2> 1566452 T3838 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard2_replica2] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370387431519\nodes_used_collection_shard2_replica2\data\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1566452 T3812 oass.SolrIndexSearcher.<init> Opening Searcher@16710f7 main
[junit4:junit4]   2> 1566453 T3812 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535\nodes_used_collection_shard2_replica1\data\tlog
[junit4:junit4]   2> 1566453 T3868 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1566454 T3812 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1566454 T3812 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1566456 T3838 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370387431519\nodes_used_collection_shard2_replica2\data\index
[junit4:junit4]   2> 1566458 T3812 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica1 state=down
[junit4:junit4]   2> 1566460 T3883 oasc.SolrCore.registerSearcher [nodes_used_collection_shard2_replica1] Registered new searcher Searcher@16710f7 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1566457 T3868 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1566461 T3838 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@19057f1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@116c37),segFN=segments_1,generation=1}
[junit4:junit4]   2> 1566463 T3838 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 1566468 T3838 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1566468 T3838 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1566471 T3838 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1566471 T3838 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1566472 T3838 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1566472 T3838 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1566472 T3838 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1566473 T3838 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1566474 T3838 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1566482 T3868 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1566489 T3838 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1566498 T3868 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1566499 T3838 oass.SolrIndexSearcher.<init> Opening Searcher@485a89 main
[junit4:junit4]   2> 1566499 T3838 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370387431519\nodes_used_collection_shard2_replica2\data\tlog
[junit4:junit4]   2> 1566501 T3838 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1566502 T3838 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1566502 T3868 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1566504 T3868 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1566504 T3868 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1566505 T3868 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1566506 T3884 oasc.SolrCore.registerSearcher [nodes_used_collection_shard2_replica2] Registered new searcher Searcher@485a89 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1566506 T3868 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1566507 T3868 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1566507 T3838 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica2 state=down
[junit4:junit4]   2> 1566507 T3868 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1566509 T3868 oasc.SolrCore.<init> [nodes_used_collection_shard1_replica2] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370387434673\nodes_used_collection_shard1_replica2\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370387434673\nodes_used_collection_shard1_replica2\data\
[junit4:junit4]   2> 1566509 T3868 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@ccb229
[junit4:junit4]   2> 1566510 T3868 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1566511 T3868 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370387434673\nodes_used_collection_shard1_replica2\data\
[junit4:junit4]   2> 1566511 T3868 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370387434673\nodes_used_collection_shard1_replica2\data\index/
[junit4:junit4]   2> 1566511 T3868 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard1_replica2] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370387434673\nodes_used_collection_shard1_replica2\data\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1566512 T3868 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370387434673\nodes_used_collection_shard1_replica2\data\index
[junit4:junit4]   2> 1566517 T3868 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1ac40a lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b9390e),segFN=segments_1,generation=1}
[junit4:junit4]   2> 1566517 T3868 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 1566522 T3868 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1566522 T3868 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1566523 T3868 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1566523 T3868 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1566524 T3868 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1566524 T3868 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1566524 T3868 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1566525 T3868 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1566525 T3868 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1566535 T3868 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1566542 T3868 oass.SolrIndexSearcher.<init> Opening Searcher@152cc03 main
[junit4:junit4]   2> 1566542 T3868 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370387434673\nodes_used_collection_shard1_replica2\data\tlog
[junit4:junit4]   2> 1566543 T3868 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1566543 T3868 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1566558 T3885 oasc.SolrCore.registerSearcher [nodes_used_collection_shard1_replica2] Registered new searcher Searcher@152cc03 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1566559 T3868 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica2 state=down
[junit4:junit4]   2> 1566868 T3817 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 1566869 T3817 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard1_replica1",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:53461_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:53461"}
[junit4:junit4]   2> 1566869 T3817 oasc.Overseer$ClusterStateUpdater.createCollection Create collection nodes_used_collection with numShards 2
[junit4:junit4]   2> 1566885 T3817 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard2_replica1",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:53431_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:53431"}
[junit4:junit4]   2> 1566893 T3817 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard2_replica2",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:53452_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:53452"}
[junit4:junit4]   2> 1566899 T3817 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard1_replica2",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:53470_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:53470"}
[junit4:junit4]   2> 1566907 T3834 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1566907 T3876 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1566908 T3862 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1566908 T3848 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1566907 T3821 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1566909 T3816 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1567152 T3857 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard1_replica1
[junit4:junit4]   2> 1567152 T3857 oasc.ZkController.register Register replica - core:nodes_used_collection_shard1_replica1 address:http://127.0.0.1:53461 collection:nodes_used_collection shard:shard1
[junit4:junit4]   2> 1567153 T3857 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leader_elect/shard1/election
[junit4:junit4]   2> 1567166 T3857 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 1567171 T3857 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up: total=2 found=1 timeoutin=180000
[junit4:junit4]   2> 1567262 T3812 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 1567262 T3812 oasc.ZkController.register Register replica - core:nodes_used_collection_shard2_replica1 address:http://127.0.0.1:53431 collection:nodes_used_collection shard:shard2
[junit4:junit4]   2> 1567263 T3812 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leader_elect/shard2/election
[junit4:junit4]   2> 1567276 T3812 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 1567281 T3812 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up: total=2 found=1 timeoutin=179999
[junit4:junit4]   2> 1567308 T3838 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard2_replica2
[junit4:junit4]   2> 1567308 T3838 oasc.ZkController.register Register replica - core:nodes_used_collection_shard2_replica2 address:http://127.0.0.1:53452 collection:nodes_used_collection shard:shard2
[junit4:junit4]   2> 1567358 T3868 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 1567358 T3868 oasc.ZkController.register Register replica - core:nodes_used_collection_shard1_replica2 address:http://127.0.0.1:53470 collection:nodes_used_collection shard:shard1
[junit4:junit4]   2> 1567571 T3857 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 1567571 T3857 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 1567571 T3857 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:53461/nodes_used_collection_shard1_replica1/
[junit4:junit4]   2> 1567572 T3857 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard1_replica1 url=http://127.0.0.1:53461 START replicas=[http://127.0.0.1:53470/nodes_used_collection_shard1_replica2/] nUpdates=100
[junit4:junit4]   2> 1567572 T3857 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard1_replica1 url=http://127.0.0.1:53461 DONE.  We have no versions.  sync failed.
[junit4:junit4]   2> 1567573 T3857 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next canidate
[junit4:junit4]   2> 1567573 T3857 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4:junit4]   2> 1567573 T3857 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:53461/nodes_used_collection_shard1_replica1/
[junit4:junit4]   2> 1567574 T3857 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leaders/shard1
[junit4:junit4]   2> ASYNC  NEW_CORE C8687 name=nodes_used_collection_shard1_replica2 org.apache.solr.core.SolrCore@18a5f0 url=http://127.0.0.1:53470/nodes_used_collection_shard1_replica2 node=127.0.0.1:53470_ C8687_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica2 props:{shard=shard1, state=down, core=nodes_used_collection_shard1_replica2, collection=nodes_used_collection, node_name=127.0.0.1:53470_, base_url=http://127.0.0.1:53470}
[junit4:junit4]   2> 1567599 T3869 C8687 P53470 oasc.SolrCore.execute [nodes_used_collection_shard1_replica2] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=16 
[junit4:junit4]   2> 1567680 T3812 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 1567680 T3812 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 1567681 T3812 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:53431/nodes_used_collection_shard2_replica1/
[junit4:junit4]   2> 1567681 T3812 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard2_replica1 url=http://127.0.0.1:53431 START replicas=[http://127.0.0.1:53452/nodes_used_collection_shard2_replica2/] nUpdates=100
[junit4:junit4]   2> 1567682 T3812 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard2_replica1 url=http://127.0.0.1:53431 DONE.  We have no versions.  sync failed.
[junit4:junit4]   2> 1567682 T3812 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next canidate
[junit4:junit4]   2> 1567682 T3812 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4:junit4]   2> 1567683 T3812 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:53431/nodes_used_collection_shard2_replica1/
[junit4:junit4]   2> 1567683 T3812 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leaders/shard2
[junit4:junit4]   2> ASYNC  NEW_CORE C8688 name=nodes_used_collection_shard2_replica2 org.apache.solr.core.SolrCore@19148a6 url=http://127.0.0.1:53452/nodes_used_collection_shard2_replica2 node=127.0.0.1:53452_ C8688_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica2 props:{shard=shard2, state=down, core=nodes_used_collection_shard2_replica2, collection=nodes_used_collection, node_name=127.0.0.1:53452_, base_url=http://127.0.0.1:53452}
[junit4:junit4]   2> 1567693 T3841 C8688 P53452 oasc.SolrCore.execute [nodes_used_collection_shard2_replica2] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1 
[junit4:junit4]   2> 1568087 T3817 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 1568119 T3821 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1568120 T3834 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1568120 T3876 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1568120 T3862 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1568120 T3848 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1568120 T3816 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1568124 T3868 oasc.ZkController.register We are http://127.0.0.1:53470/nodes_used_collection_shard1_replica2/ and leader is http://127.0.0.1:53461/nodes_used_collection_shard1_replica1/
[junit4:junit4]   2> 1568124 T3868 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard1_replica2 baseURL=http://127.0.0.1:53470
[junit4:junit4]   2> 1568125 T3868 oasc.ZkController.checkRecovery Core needs to recover:nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 1568125 T3868 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> 1568126 T3888 C8687 P53470 oasc.RecoveryStrategy.run Starting recovery process.  core=nodes_used_collection_shard1_replica2 recoveringAfterStartup=true
[junit4:junit4]   2> 1568127 T3888 C8687 P53470 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 1568127 T3888 C8687 P53470 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica2 state=recovering
[junit4:junit4]   2> 1568129 T3868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 1568131 T3888 C8687 P53470 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 1568134 T3868 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard1&name=nodes_used_collection_shard1_replica2&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=3416 
[junit4:junit4]   2> 1568136 T3857 oasc.ZkController.register We are http://127.0.0.1:53461/nodes_used_collection_shard1_replica1/ and leader is http://127.0.0.1:53461/nodes_used_collection_shard1_replica1/
[junit4:junit4]   2> 1568136 T3857 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard1_replica1 baseURL=http://127.0.0.1:53461
[junit4:junit4]   2> 1568137 T3857 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 1568137 T3857 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica1 state=active
[junit4:junit4]   2> 1568143 T3857 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 1568145 T3857 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard1&name=nodes_used_collection_shard1_replica1&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=3428 
[junit4:junit4]   2> 1568145 T3858 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:53470__nodes_used_collection_shard1_replica2, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4]   2> 1568150 T3838 oasc.ZkController.register We are http://127.0.0.1:53452/nodes_used_collection_shard2_replica2/ and leader is http://127.0.0.1:53431/nodes_used_collection_shard2_replica1/
[junit4:junit4]   2> 1568150 T3838 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard2_replica2 baseURL=http://127.0.0.1:53452
[junit4:junit4]   2> 1568151 T3838 oasc.ZkController.checkRecovery Core needs to recover:nodes_used_collection_shard2_replica2
[junit4:junit4]   2> 1568151 T3838 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> 1568152 T3889 C8688 P53452 oasc.RecoveryStrategy.run Starting recovery process.  core=nodes_used_collection_shard2_replica2 recoveringAfterStartup=true
[junit4:junit4]   2> 1568152 T3889 C8688 P53452 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 1568152 T3838 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 1568153 T3889 C8688 P53452 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica2 state=recovering
[junit4:junit4]   2> 1568155 T3838 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard2&name=nodes_used_collection_shard2_replica2&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=3436 
[junit4:junit4]   2> 1568156 T3889 C8688 P53452 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 1568164 T3807 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:53452__nodes_used_collection_shard2_replica2, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4]   2> 1568181 T3812 oasc.ZkController.register We are http://127.0.0.1:53431/nodes_used_collection_shard2_replica1/ and leader is http://127.0.0.1:53431/nodes_used_collection_shard2_replica1/
[junit4:junit4]   2> 1568181 T3812 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard2_replica1 baseURL=http://127.0.0.1:53431
[junit4:junit4]   2> 1568181 T3812 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 1568181 T3812 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica1 state=active
[junit4:junit4]   2> 1568185 T3812 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 1568186 T3812 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard2&name=nodes_used_collection_shard2_replica1&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=3469 
[junit4:junit4]   2> 1568188 T3818 oasc.OverseerCollectionProcessor.createCollection Finished create command on all shards for collection: nodes_used_collection
[junit4:junit4]   2> 1568197 T3834 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work/qnr-0000000000 state: SyncConnected type NodeDataChanged
[junit4:junit4]   2> 1568200 T3816 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4]   2> 1568202 T3828 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={numShards=2&name=nodes_used_collection&replicationFactor=2&action=CREATE&wt=javabin&version=2} status=0 QTime=3498 
[junit4:junit4]   2> 1568203 T3818 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Message id:/overseer/collection-queue-work/qn-0000000000 complete, response:{success={null={responseHeader={status=0,QTime=3416},core=nodes_used_collection_shard1_replica2},null={responseHeader={status=0,QTime=3428},core=nodes_used_collection_shard1_replica1},null={responseHeader={status=0,QTime=3436},core=nodes_used_collection_shard2_replica2},null={responseHeader={status=0,QTime=3469},core=nodes_used_collection_shard2_replica1}}}
[junit4:junit4]   2> 1568204 T3794 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 1568205 T3794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1568209 T3891 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7ae50c name:ZooKeeperConnection Watcher:127.0.0.1:53424/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1568209 T3794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1568211 T3794 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1568215 T3794 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 1568218 T3794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1568223 T3893 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@199b715 name:ZooKeeperConnection Watcher:127.0.0.1:53424/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1568223 T3794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1568225 T3794 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1568232 T3809 oasha.CollectionsHandler.handleCreateAction Creating Collection : numShards=1&maxShardsPerNode=1&name=awholynewcollection_0&replicationFactor=2&action=CREATE&wt=javabin&version=2
[junit4:junit4]   2> 1568236 T3816 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4]   2> 1568239 T3818 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000002 message:{
[junit4:junit4]   2> 	  "operation":"createcollection",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "maxShardsPerNode":"1",
[junit4:junit4]   2> 	  "createNodeSet":null,
[junit4:junit4]   2> 	  "name":"awholynewcollection_0",
[junit4:junit4]   2> 	  "replicationFactor":"2"}
[junit4:junit4]   2> 1568239 T3818 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica1 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:53431_
[junit4:junit4]   2> 1568239 T3818 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica2 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:53443_
[junit4:junit4]   2> 1568241 T3808 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535\awholynewcollection_0_shard1_replica1
[junit4:junit4]   2> 1568241 T3808 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 1568241 T3830 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica2' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370387429575\awholynewcollection_0_shard1_replica2
[junit4:junit4]   2> 1568241 T3830 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 1568244 T3808 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4]   2> 1568244 T3830 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4]   2> 1568244 T3830 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1568244 T3808 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1568246 T3808 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 1568246 T3830 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 1568247 T3830 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4]   2> 1568246 T3808 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4]   2> 1568252 T3808 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 1568253 T3830 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 1568255 T3808 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535\awholynewcollection_0_shard1_replica1\'
[junit4:junit4]   2> 1568260 T3830 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370387429575\awholynewcollection_0_shard1_replica2\'
[junit4:junit4]   2> 1568335 T3830 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1568340 T3808 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1568410 T3830 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1568415 T3830 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1568418 T3808 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1568420 T3830 oass.IndexSchema.readSchema [awholynewcollection_0_shard1_replica2] Schema name=test
[junit4:junit4]   2> 1568422 T3808 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1568427 T3808 oass.IndexSchema.readSchema [awholynewcollection_0_shard1_replica1] Schema name=test
[junit4:junit4]   2> 1569143 T3808 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1569154 T3808 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1569158 T3808 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1569161 T3830 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1569171 T3830 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1569176 T3830 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1569190 T3808 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1569197 T3808 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1569203 T3830 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1569204 T3808 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1569206 T3808 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1569206 T3808 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1569206 T3808 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1569208 T3808 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1569208 T3808 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1569209 T3808 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1569209 T3808 oasc.SolrCore.<init> [awholynewcollection_0_shard1_replica1] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535\awholynewcollection_0_shard1_replica1\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535\awholynewcollection_0_shard1_replica1\data\
[junit4:junit4]   2> 1569209 T3830 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1569209 T3808 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@ccb229
[junit4:junit4]   2> 1569210 T3808 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1569211 T3808 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535\awholynewcollection_0_shard1_replica1\data\
[junit4:junit4]   2> 1569211 T3808 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535\awholynewcollection_0_shard1_replica1\data\index/
[junit4:junit4]   2> 1569212 T3808 oasc.SolrCore.initIndex WARN [awholynewcollection_0_shard1_replica1] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535\awholynewcollection_0_shard1_replica1\data\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1569213 T3808 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535\awholynewcollection_0_shard1_replica1\data\index
[junit4:junit4]   2> 1569215 T3830 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1569217 T3830 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1569218 T3830 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1569218 T3830 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1569220 T3830 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1569220 T3830 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1569220 T3808 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@19fb36f lockFactory=org.apache.lucene.store.NativeFSLockFactory@1d6e134),segFN=segments_1,generation=1}
[junit4:junit4]   2> 1569220 T3808 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 1569220 T3830 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1569221 T3830 oasc.SolrCore.<init> [awholynewcollection_0_shard1_replica2] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370387429575\awholynewcollection_0_shard1_replica2\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370387429575\awholynewcollection_0_shard1_replica2\data\
[junit4:junit4]   2> 1569221 T3830 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@ccb229
[junit4:junit4]   2> 1569222 T3830 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1569223 T3830 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370387429575\awholynewcollection_0_shard1_replica2\data\
[junit4:junit4]   2> 1569223 T3830 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370387429575\awholynewcollection_0_shard1_replica2\data\index/
[junit4:junit4]   2> 1569224 T3830 oasc.SolrCore.initIndex WARN [awholynewcollection_0_shard1_replica2] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370387429575\awholynewcollection_0_shard1_replica2\data\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1569225 T3830 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370387429575\awholynewcollection_0_shard1_replica2\data\index
[junit4:junit4]   2> 1569225 T3808 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1569225 T3808 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1569226 T3808 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1569227 T3808 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1569227 T3808 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1569227 T3808 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1569228 T3808 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1569229 T3808 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1569229 T3808 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1569230 T3830 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7bfe42 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5f0117),segFN=segments_1,generation=1}
[junit4:junit4]   2> 1569231 T3830 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 1569235 T3830 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1569236 T3830 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1569236 T3830 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1569237 T3830 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1569238 T3830 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1569238 T3830 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1569238 T3830 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1569239 T3830 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1569239 T3830 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1569244 T3830 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1569245 T3817 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 1569246 T3817 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:53470__nodes_used_collection_shard1_replica2",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard1_replica2",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:53470_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:53470"}
[junit4:junit4]   2> 1569250 T3808 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1569254 T3808 oass.SolrIndexSearcher.<init> Opening Searcher@1e09019 main
[junit4:junit4]   2> 1569255 T3830 oass.SolrIndexSearcher.<init> Opening Searcher@64fc2f main
[junit4:junit4]   2> 1569256 T3808 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535\awholynewcollection_0_shard1_replica1\data\tlog
[junit4:junit4]   2> 1569256 T3830 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370387429575\awholynewcollection_0_shard1_replica2\data\tlog
[junit4:junit4]   2> 1569259 T3830 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1569260 T3830 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1569259 T3808 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1569260 T3808 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1569263 T3817 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:53461__nodes_used_collection_shard1_replica1",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard1_replica1",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:53461_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:53461"}
[junit4:junit4]   2> 1569264 T3895 oasc.SolrCore.registerSearcher [awholynewcollection_0_shard1_replica2] Registered new searcher Searcher@64fc2f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1569266 T3830 oasc.ZkController.publish publishing core=awholynewcollection_0_shard1_replica2 state=down
[junit4:junit4]   2> 1569267 T3808 oasc.ZkController.publish publishing core=awholynewcollection_0_shard1_replica1 state=down
[junit4:junit4]   2> 1569267 T3894 oasc.SolrCore.registerSearcher [awholynewcollection_0_shard1_replica1] Registered new searcher Searcher@1e09019 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1569274 T3817 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:53452__nodes_used_collection_shard2_replica2",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard2_replica2",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:53452_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:53452"}
[junit4:junit4]   2> 1569280 T3817 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:53431__nodes_used_collection_shard2_replica1",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard2_replica1",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:53431_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:53431"}
[junit4:junit4]   2> 1569286 T3817 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"awholynewcollection_0_shard1_replica2",
[junit4:junit4]   2> 	  "collection":"awholynewcollection_0",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:53443_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:53443"}
[junit4:junit4]   2> 1569286 T3817 oasc.Overseer$ClusterStateUpdater.createCollection Create collection awholynewcollection_0 with numShards 1
[junit4:junit4]   2> 1569291 T3817 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"awholynewcollection_0_shard1_replica1",
[junit4:junit4]   2> 	  "collection":"awholynewcollection_0",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:53431_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:53431"}
[junit4:junit4]   2> 1569299 T3834 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1569299 T3876 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1569300 T3862 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1569300 T3848 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1569300 T3816 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1569300 T3893 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1569300 T3821 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1569300 T3891 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1569587 T3858 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:53470__nodes_used_collection_shard1_replica2, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4]   2> 1569587 T3858 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:53470__nodes_used_collection_shard1_replica2&state=recovering&nodeName=127.0.0.1:53470_&action=PREPRECOVERY&checkLive=true&core=nodes_used_collection_shard1_replica1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1442 
[junit4:junit4]   2> 1569604 T3807 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:53452__nodes_used_collection_shard2_replica2, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4]   2> 1569604 T3807 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:53452__nodes_used_collection_shard2_replica2&state=recovering&nodeName=127.0.0.1:53452_&action=PREPRECOVERY&checkLive=true&core=nodes_used_collection_shard2_replica1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1440 
[junit4:junit4]   2> 1569988 T3808 oasc.CoreContainer.registerCore registering core: awholynewcollection_0_shard1_replica1
[junit4:junit4]   2> 1569988 T3808 oasc.ZkController.register Register replica - core:awholynewcollection_0_shard1_replica1 address:http://127.0.0.1:53431 collection:awholynewcollection_0 shard:shard1
[junit4:junit4]   2> 1569988 T3830 oasc.CoreContainer.registerCore registering core: awholynewcollection_0_shard1_replica2
[junit4:junit4]   2> 1569988 T3830 oasc.ZkController.register Register replica - core:awholynewcollection_0_shard1_replica2 address:http://127.0.0.1:53443 collection:awholynewcollection_0 shard:shard1
[junit4:junit4]   2> 1569989 T3808 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0/leader_elect/shard1/election
[junit4:junit4]   2> 1569990 T3830 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0/leader_elect/shard1/election
[junit4:junit4]   2> 1570010 T3808 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 1570017 T3808 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 1570017 T3808 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 1570018 T3808 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:53431/awholynewcollection_0_shard1_replica1/
[junit4:junit4]   2> 1570018 T3808 oasu.PeerSync.sync PeerSync: core=awholynewcollection_0_shard1_replica1 url=http://127.0.0.1:53431 START replicas=[http://127.0.0.1:53443/awholynewcollection_0_shard1_replica2/] nUpdates=100
[junit4:junit4]   2> 1570018 T3808 oasu.PeerSync.sync PeerSync: core=awholynewcollection_0_shard1_replica1 url=http://127.0.0.1:53431 DONE.  We have no versions.  sync failed.
[junit4:junit4]   2> 1570018 T3808 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next canidate
[junit4:junit4]   2> 1570019 T3808 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4:junit4]   2> 1570019 T3808 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:53431/awholynewcollection_0_shard1_replica1/
[junit4:junit4]   2> 1570020 T3808 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0/leaders/shard1
[junit4:junit4]   2> ASYNC  NEW_CORE C8689 name=awholynewcollection_0_shard1_replica2 org.apache.solr.core.SolrCore@582113 url=http://127.0.0.1:53443/awholynewcollection_0_shard1_replica2 node=127.0.0.1:53443_ C8689_STATE=coll:awholynewcollection_0 core:awholynewcollection_0_shard1_replica2 props:{shard=shard1, state=down, core=awholynewcollection_0_shard1_replica2, collection=awholynewcollection_0, node_name=127.0.0.1:53443_, base_url=http://127.0.0.1:53443}
[junit4:junit4]   2> 1570031 T3829 C8689 P53443 oasc.SolrCore.execute [awholynewcollection_0_shard1_replica2] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1 
[junit4:junit4]   2> 1570402 T3817 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 1570420 T3834 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1570421 T3876 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1570421 T3862 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1570421 T3893 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1570421 T3848 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1570421 T3891 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1570422 T3821 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1570422 T3816 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1570437 T3808 oasc.ZkController.register We are http://127.0.0.1:53431/awholynewcollection_0_shard1_replica1/ and leader is http://127.0.0.1:53431/awholynewcollection_0_shard1_replica1/
[junit4:junit4]   2> 1570437 T3808 oasc.ZkController.register No LogReplay needed for core=awholynewcollection_0_shard1_replica1 baseURL=http://127.0.0.1:53431
[junit4:junit4]   2> 1570437 T3808 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 1570437 T3808 oasc.ZkController.publish publishing core=awholynewcollection_0_shard1_replica1 state=active
[junit4:junit4]   2> 1570441 T3808 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 1570442 T3808 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=1&shard=shard1&name=awholynewcollection_0_shard1_replica1&action=CREATE&collection=awholynewcollection_0&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=2201 
[junit4:junit4]   2> 1570730 T3830 oasc.ZkController.register We are http://127.0.0.1:53443/awholynewcollection_0_shard1_replica2/ and leader is http://127.0.0.1:53431/awholynewcollection_0_shard1_replica1/
[junit4:junit4]   2> 1570730 T3830 oasc.ZkController.register No LogReplay needed for core=awholynewcollection_0_shard1_replica2 baseURL=http://127.0.0.1:53443
[junit4:junit4]   2> 1570730 T3830 oasc.ZkController.checkRecovery Core needs to recover:awholynewcollection_0_shard1_replica2
[junit4:junit4]   2> 1570731 T3830 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> 1570732 T3897 C8689 P53443 oasc.RecoveryStrategy.run Starting recovery process.  core=awholynewcollection_0_shard1_replica2 recoveringAfterStartup=true
[junit4:junit4]   2> 1570732 T3897 C8689 P53443 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 1570733 T3897 C8689 P53443 oasc.ZkController.publish publishing core=awholynewcollection_0_shard1_replica2 state=recovering
[junit4:junit4]   2> 1570733 T3830 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 1570737 T3897 C8689 P53443 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 1570738 T3830 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=1&shard=shard1&name=awholynewcollection_0_shard1_replica2&action=CREATE&collection=awholynewcollection_0&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=2497 
[junit4:junit4]   2> 1570739 T3818 oasc.OverseerCollectionProcessor.createCollection Finished create command on all shards for collection: awholynewcollection_0
[junit4:junit4]   2> 1570744 T3816 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work/qnr-0000000002 state: SyncConnected type NodeDataChanged
[junit4:junit4]   2> 1570748 T3816 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4]   2> 1570749 T3810 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:53443__awholynewcollection_0_shard1_replica2, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4]   2> 1570749 T3818 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Message id:/overseer/collection-queue-work/qn-0000000002 complete, response:{success={null={responseHeader={status=0,QTime=2201},core=awholynewcollection_0_shard1_replica1},null={responseHeader={status=0,QTime=2497},core=awholynewcollection_0_shard1_replica2}}}
[junit4:junit4]   2> 1570751 T3809 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={numShards=1&maxShardsPerNode=1&name=awholynewcollection_0&replicationFactor=2&action=CREATE&wt=javabin&version=2} status=0 QTime=2519 
[junit4:junit4]   2> 1570753 T3794 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 1570753 T3796 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13f1174b3fe000e, likely client has closed socket
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 
[junit4:junit4]   2> 1570756 T3794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1570759 T3899 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@31b33a name:ZooKeeperConnection Watcher:127.0.0.1:53424/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1570759 T3794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1570761 T3794 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1570768 T3811 oasha.CollectionsHandler.handleCreateAction Creating Collection : numShards=5&maxShardsPerNode=4&name=awholynewcollection_1&replicationFactor=3&action=CREATE&wt=javabin&version=2
[junit4:junit4]   2> 1570771 T3816 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4]   2> 1570774 T3818 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000004 message:{
[junit4:junit4]   2> 	  "operation":"createcollection",
[junit4:junit4]   2> 	  "numShards":"5",
[junit4:junit4]   2> 	  "maxShardsPerNode":"4",
[junit4:junit4]   2> 	  "createNodeSet":null,
[junit4:junit4]   2> 	  "name":"awholynewcollection_1",
[junit4:junit4]   2> 	  "replicationFactor":"3"}
[junit4:junit4]   2> 1570774 T3818 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_1_shard1_replica1 as part of slice shard1 of collection awholynewcollection_1 on 127.0.0.1:53461_
[junit4:junit4]   2> 1570775 T3818 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_1_shard1_replica2 as part of slice shard1 of collection awholynewcollection_1 on 127.0.0.1:53431_
[junit4:junit4]   2> 1570775 T3818 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_1_shard1_replica3 as part of slice shard1 of collection awholynewcollection_1 on 127.0.0.1:53443_
[junit4:junit4]   2> 1570775 T3818 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_1_shard2_replica1 as part of slice shard2 of collection awholynewcollection_1 on 127.0.0.1:53452_
[junit4:junit4]   2> 1570775 T3818 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_1_shard2_replica2 as part of slice shard2 of collection awholynewcollection_1 on 127.0.0.1:53470_
[junit4:junit4]   2> 1570775 T3818 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_1_shard2_replica3 as part of slice shard2 of collection awholynewcollection_1 on 127.0.0.1:53461_
[junit4:junit4]   2> 1570776 T3825 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_1_shard1_replica3' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370387429575\awholynewcollection_1_shard1_replica3
[junit4:junit4]   2> 1570776 T3825 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_1
[junit4:junit4]   2> 1570777 T3838 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_1_shard2_replica1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370387431519\awholynewcollection_1_shard2_replica1
[junit4:junit4]   2> 1570777 T3838 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_1
[junit4:junit4]   2> 1570778 T3825 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_1
[junit4:junit4]   2> 1570778 T3825 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1570779 T3838 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_1
[junit4:junit4]   2> 1570779 T3838 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1570776 T3854 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_1_shard1_replica1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370387433206\awholynewcollection_1_shard1_replica1
[junit4:junit4]   2> 1570781 T3854 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_1
[junit4:junit4]   2> 1570778 T3868 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_1_shard2_replica2' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370387434673\awholynewcollection_1_shard2_replica2
[junit4:junit4]   2> 1570782 T3868 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_1
[junit4:junit4]   2> 1570777 T3818 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_1_shard3_replica1 as part of slice shard3 of collection awholynewcollection_1 on 127.0.0.1:53431_
[junit4:junit4]   2> 1570783 T3854 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_1
[junit4:junit4]   2> 1570783 T3854 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1570777 T3808 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_1_shard1_replica2' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535\awholynewcollection_1_shard1_replica2
[junit4:junit4]   2> 1570784 T3808 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_1
[junit4:junit4]   2> 1570784 T3838 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 1570784 T3838 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_1
[junit4:junit4]   2> 1570784 T3868 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_1
[junit4:junit4]   2> 1570786 T3868 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1570786 T3808 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_1
[junit4:junit4]   2> 1570786 T3808 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1570783 T3818 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_1_shard3_replica2 as part of slice shard3 of collection awholynewcollection_1 on 127.0.0.1:53443_
[junit4:junit4]   2> 1570787 T3818 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_1_shard3_replica3 as part of slice shard3 of collection awholynewcollection_1 on 127.0.0.1:53452_
[junit4:junit4]   2> 1570782 T3825 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 1570787 T3825 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_1
[junit4:junit4]   2> 1570788 T3818 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_1_shard4_replica1 as part of slice shard4 of collection awholynewcollection_1 on 127.0.0.1:53470_
[junit4:junit4]   2> 1570782 T3855 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_1_shard2_replica3' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370387433206\awholynewcollection_1_shard2_replica3
[junit4:junit4]   2> 1570789 T3855 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_1
[junit4:junit4]   2> 1570789 T3868 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 1570790 T3868 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_1
[junit4:junit4]   2> 1570786 T3854 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 1570791 T3854 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_1
[junit4:junit4]   2> 1570786 T3807 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_1_shard3_replica1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535\awholynewcollection_1_shard3_replica1
[junit4:junit4]   2> 1570798 T3807 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_1
[junit4:junit4]   2> 1570797 T3838 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_1
[junit4:junit4]   2> 1570791 T3818 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_1_shard4_replica2 as part of slice shard4 of collection awholynewcollection_1 on 127.0.0.1:53461_
[junit4:junit4]   2> 1570800 T3818 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_1_shard4_replica3 as part of slice shard4 of collection awholynewcollection_1 on 127.0.0.1:53431_
[junit4:junit4]   2> 1570800 T3825 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_1
[junit4:junit4]   2> 1570796 T3855 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 1570802 T3855 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_1
[junit4:junit4]   2> 1570796 T3842 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_1_shard3_replica3' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370387431519\awholynewcollection_1_shard3_replica3
[junit4:junit4]   2> 1570803 T3842 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_1
[junit4:junit4]   2> 1570803 T3868 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_1
[junit4:junit4]   2> 1570793 T3808 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 1570804 T3808 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_1
[junit4:junit4]   2> 1570793 T3826 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_1_shard3_replica2' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370387429575\awholynewcollection_1_shard3_replica2
[junit4:junit4]   2> 1570805 T3826 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_1
[junit4:junit4]   2> 1570803 T3858 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_1_shard4_replica2' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370387433206\awholynewcollection_1_shard4_replica2
[junit4:junit4]   2> 1570806 T3858 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_1
[junit4:junit4]   2> 1570806 T3809 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_1_shard4_replica3' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535\awholynewcollection_1_shard4_replica3
[junit4:junit4]   2> 1570808 T3809 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_1
[junit4:junit4]   2> 1570809 T3838 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370387431519\awholynewcollection_1_shard2_replica1\'
[junit4:junit4]   2> 1570810 T3842 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 1570810 T3842 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_1
[junit4:junit4]   2> 1570800 T3818 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_1_shard5_replica1 as part of slice shard5 of collection awholynewcollection_1 on 127.0.0.1:53443_
[junit4:junit4]   2> 1570800 T3871 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_1_shard4_replica1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370387434673\awholynewcollection_1_shard4_replica1
[junit4:junit4]   2> 1570812 T3871 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_1
[junit4:junit4]   2> 1570817 T3818 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_1_shard5_replica2 as part of slice shard5 of collection awholynewcollection_1 on 127.0.0.1:53452_
[junit4:junit4]   2> 1570817 T3871 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 1570817 T3871 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_1
[junit4:junit4]   2> 1570817 T3818 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_1_shard5_replica3 as part of slice shard5 of collection awholynewcollection_1 on 127.0.0.1:53470_
[junit4:junit4]   2> 1570807 T3854 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_1
[junit4:junit4]   2> 1570805 T3807 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 1570821 T3807 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_1
[junit4:junit4]   2> 1570820 T3855 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370387433206\awholynewcollection_1_shard2_replica3\'
[junit4:junit4]   2> 1570824 T3868 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370387434673\awholynewcollection_1_shard2_replica2\'
[junit4:junit4]   2> 1570819 T3825 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370387429575\awholynewcollection_1_shard1_replica3\'
[junit4:junit4]   2> 1570817 T3826 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 1570830 T3826 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_1
[junit4:junit4]   2> 1570831 T3842 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370387431519\awholynewcollection_1_shard3_replica3\'
[junit4:junit4]   2> 1570817 T3858 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 1570835 T3858 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_1
[junit4:junit4]   2> 1570835 T3827 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_1_shard5_replica1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370387429575\awholynewcollection_1_shard5_replica1
[junit4:junit4]   2> 1570835 T3827 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_1
[junit4:junit4]   2> 1570817 T3809 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 1570836 T3809 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_1
[junit4:junit4]   2> 1570835 T3843 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_1_shard5_replica2' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370387431519\awholynewcollection_1_shard5_replica2
[junit4:junit4]   2> 1570840 T3843 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_1
[junit4:junit4]   2> 1570835 T3808 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_1
[junit4:junit4]   2> 1570845 T3872 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_1_shard5_replica3' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370387434673\awholynewcollection_1_shard5_replica3
[junit4:junit4]   2> 1570846 T3872 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_1
[junit4:junit4]   2> 1570849 T3843 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 1570849 T3843 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_1
[junit4:junit4]   2> 1570851 T3807 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535\awholynewcollection_1_shard3_replica1\'
[junit4:junit4]   2> 1570856 T3826 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370387429575\awholynewcollection_1_shard3_replica2\'
[junit4:junit4]   2> 1570859 T3854 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370387433206\awholynewcollection_1_shard1_replica1\'
[junit4:junit4]   2> 1570859 T3872 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 1570860 T3872 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_1
[junit4:junit4]   2> 1570861 T3827 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 1570861 T3827 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_1
[junit4:junit4]   2> 1570862 T3871 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370387434673\awholynewcollection_1_shard4_replica1\'
[junit4:junit4]   2> 1570873 T3809 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535\awholynewcollection_1_shard4_replica3\'
[junit4:junit4]   2> 1570875 T3843 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370387431519\awholynewcollection_1_shard5_replica2\'
[junit4:junit4]   2> 1570877 T3808 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1370387428535\awholynewcollection_1_shard1_replica2\'
[junit4:junit4]   2> 1570882 T3827 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1370387429575\awholynewcollection_1_shard5_replica1\'
[junit4:junit4]   2> 1570885 T3858 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1370387433206\awholynewcollection_1_shard4_replica2\'
[junit4:junit4]   2> 1570885 T3872 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370387434673\awholynewcollection_1_shard5_replica3\'
[junit4:junit4]   2> 1570915 T3838 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1570930 T3807 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1571004 T3826 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1571032 T3807 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> ASYNC  NEW_CORE C8690 name=nodes_used_collection_shard2_replica2 org.apache.solr.core.SolrCore@19148a6 url=http://127.0.0.1:53452/nodes_used_collection_shard2_replica2 node=127.0.0.1:53452_ C8690_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica2 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica2, collection=nodes_used_collection, node_name=127.0.0.1:53452_, base_url=http://127.0.0.1:53452}
[junit4:junit4]   2> 1571051 T3889 C8690 P53452 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:53431/nodes_used_collection_shard2_replica1/ core=nodes_used_collection_shard2_replica2 - recoveringAfterStartup=true
[junit4:junit4]   2> 1571051 T3889 C8690 P53452 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard2_replica2 url=http://127.0.0.1:53452 START replicas=[http://127.0.0.1:53431/nodes_used_collection_shard2_replica1/] nUpdates=100
[junit4:junit4]   2> 1571059 T3826 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1571061 T3889 C8690 P53452 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 1571061 T3889 C8690 P53452 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=nodes_used_collection_shard2_replica2
[junit4:junit4]   2> 1571061 T3889 C8690 P53452 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=nodes_used_collection_shard2_replica2
[junit4:junit4]   2> 1571061 T3889 C8690 P53452 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=nodes_used_collection_shard2_replica2
[junit4:junit4]   2> 1571061 T3889 C8690 P53452 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 1571062 T3889 C8690 P53452 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:53431/nodes_used_collection_shard2_replica1/. core=nodes_used_collection_shard2_replica2
[junit4:junit4]   2> 1571062 T3889 C8690 P53452 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C8691 name=nodes_used_collection_shard1_replica2 org.apache.solr.core.SolrCore@18a5f0 url=http://127.0.0.1:53470/nodes_used_collection_shard1_replica2 node=127.0.0.1:53470_ C8691_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica2 props:{shard=shard1, state=recovering, core=nodes_used_collection_shard1_replica2, collection=nodes_used_collection, node_name=127.0.0.1:53470_, base_url=http://127.0.0.1:53470}
[junit4:junit4]   2> 1571084 T3888 C8691 P53470 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:53461/nodes_used_collection_shard1_replica1/ core=nodes_used_collection_shard1_replica2 - recoveringAfterStartup=true
[junit4:junit4]   2> 1571085 T3888 C8691 P53470 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard1_replica2 url=http://127.0.0.1:53470 START replicas=[http://127.0.0.1:53461/nodes_used_collection_shard1_replica1/] nUpdates=100
[junit4:junit4]   2> 1571086 T3888 C8691 P53470 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 1571086 T3888 C8691 P53470 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 1571086 T3888 C8691 P53470 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 1571086 T3888 C8691 P53470 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 1571086 T3888 C8691 P53470 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 1571087 T3888 C8691 P53470 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:53461/nodes_used_collection_shard1_replica1/. core=nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 1571087 T3888 C8691 P53470 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 1571090 T3838 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> ASYNC  NEW_CORE C8692 name=nodes_used_collection_shard1_replica1 org.apache.solr.core.SolrCore@1795835 url=http://127.0.0.1:53461/nodes_used_collection_shard1_replica1 node=127.0.0.1:53461_ C8692_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica1 props:{shard=shard1, state=active, core=nodes_used_collection_shard1_replica1, collection=nodes_used_collection, node_name=127.0.0.1:53461_, base_url=http://127.0.0.1:53461, leader=true}
[junit4:junit4]   2> 1571100 T3856 C8692 P53461 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 1571109 T3826 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1571110 T3807 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1571116 T3807 oass.IndexSchema.readSchema [awholynewcollection_1_shard3_replica1] Schema name=test
[junit4:junit4]   2> 1571128 T3856 C8692 P53461 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1aea164 lockFactory=org.apache.lucene.store.NativeFSLockFactory@187bc5e),segFN=segments_1,generation=1}
[junit4:junit4]   2> 1571133 T3826 oass.IndexSchema.readSchema [awholynewcollection_1_shard3_replica2] Schema name=test
[junit4:junit4]   2> 1571161 T3868 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> ASYNC  NEW_CORE C8693 name=nodes_used_collection_shard1_replica1 org.apache.solr.core.SolrCore@1795835 url=http://127.0.0.1:53461/nodes_used_collection_shard1_replica1 node=127.0.0.1:53461_ C8693_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica1 props:{shard=shard1, state=active, core=nodes_used_collection_shard1_replica1, collection=nodes_used_collection, node_name=127.0.0.1:53461_, base_url=http://127.0.0.1:53461, leader=true}
[junit4:junit4]   2> 1571205 T3853 C8693 P53461 oasc.SolrCore.execute [nodes_used_collection_shard1_replica1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=95 
[junit4:junit4]   2> 1571207 T3856 C8693 P53461 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 1571208 T3856 C8693 P53461 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1aea164 lockFactory=org.apache.lucene.store.NativeFSLockFactory@187bc5e),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1aea164 lockFactory=org.apache.lucene.store.NativeFSLockFactory@187bc5e),segFN=segments_2,generation=2}
[junit4:junit4]   2> 1571209 T3856 C8693 P53461 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 1571209 T3838 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1571210 T3856 C8693 P53461 oass.SolrIndexSearcher.<init> Opening Searcher@f85733 realtime
[junit4:junit4]   2> 1571211 T3856 C8693 P53461 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 1571211 T3856 C8693 P53461 oasup.LogUpdateProcessor.finish [nodes_used_collection_shard1_replica1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 111
[junit4:junit4]   2> ASYNC  NEW_CORE C8694 name=nodes_used_collection_shard1_replica2 org.apache.solr.core.SolrCore@18a5f0 url=http://127.0.0.1:53470/nodes_used_collection_shard1_replica2 node=127.0.0.1:53470_ C8694_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica2 props:{shard=shard1, state=recovering, core=nodes_used_collection_shard1_replica2, collection=nodes_used_collection, node_name=127.0.0.1:53470_, base_url=http://127.0.0.1:53470}
[junit4:junit4]   2> 1571212 T3888 C8694 P53470 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 1571212 T3888 C8694 P53470 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 1571218 T3838 oass.IndexSchema.readSchema [awholynewcollection_1_shard2_replica1] Schema name=test
[junit4:junit4]   2> 1571248 T3857 C8693 P53461 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 1571259 T3857 C8693 P53461 oasc.SolrCore.execute [nodes_used_collection_shard1_replica1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=12 
[junit4:junit4]   2> 1571259 T3888 C8694 P53470 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 1571259 T3888 C8694 P53470 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 1571259 T3888 C8694 P53470 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 1571241 T3843 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1571267 T3853 C8693 P53461 oasc.SolrCore.execute [nodes_used_collection_shard1_replica1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
[junit4:junit4]   2> 1571269 T3888 C8694 P53470 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 1571269 T3888 C8694 P53470 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370387434673\nodes_used_collection_shard1_replica2\data\index.20130604161042898
[junit4:junit4]   2> 1571271 T3888 C8694 P53470 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@783800 lockFactory=org.apache.lucene.store.NativeFSLockFactory@188c327) fullCopy=false
[junit4:junit4]   2> 1571274 T3856 C8693 P53461 oasc.SolrCore.execute [nodes_used_collection_shard1_replica1] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 1571283 T3888 C8694 P53470 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 1571284 T3888 C8694 P53470 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 1571284 T3888 C8694 P53470 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 1571294 T3888 C8694 P53470 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1ac40a lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b9390e),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1ac40a lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b9390e),segFN=segments_2,generation=2}
[junit4:junit4]   2> 1571306 T3888 C8694 P53470 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 1571314 T3888 C8694 P53470 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 1571314 T3888 C8694 P53470 oass.SolrIndexSearcher.<init> Opening Searcher@1f3957c main
[junit4:junit4]   2> 1571318 T3885 oasc.SolrCore.registerSearcher [nodes_used_collection_shard1_replica2] Registered new searcher Searcher@1f3957c main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 1571319 T3888 C8694 P53470 oasc.CachingDirectoryFactory.closeCacheValue looking to close C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370387434673\nodes_used_collection_shard1_replica2\data\index.20130604161042898 [CachedDir<<refCount=0;path=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370387434673\nodes_used_collection_shard1_replica2\data\index.20130604161042898;done=true>>]
[junit4:junit4]   2> 1571319 T3888 C8694 P53470 oasc.CachingDirectoryFactory.close Closing directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370387434673\nodes_used_collection_shard1_replica2\data\index.20130604161042898
[junit4:junit4]   2> 1571320 T3888 C8694 P53470 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1370387434673\nodes_used_collection_shard1_replica2\data\index.20130604161042898
[junit4:junit4]   2> 1571320 T3888 C8694 P53470 oasc.RecoveryStrategy.replay No replay needed. core=nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 1571320 T3888 C8694 P53470 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 1571320 T3888 C8694 P53470 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica2 state=active
[junit4:junit4]   2> 1571327 T3888 C8694 P53470 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 1571358 T3868 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1571397 T3854 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> ASYNC  NEW_CORE C8695 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@1dfe3d0 url=http://127.0.0.1:53431/nodes_used_collection_shard2_replica1 node=127.0.0.1:53431_ C8695_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=active, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:53431_, base_url=http://127.0.0.1:53431, leader=true}
[junit4:junit4]   2> 1571429 T3812 C8695 P53431 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 1571446 T3855 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> ASYNC  NEW_CORE C8696 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@1dfe3d0 url=http://127.0.0.1:53431/nodes_used_collection_shard2_replica1 node=127.0.0.1:53431_ C8696_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=active, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:53431_, base_url=http://127.0.0.1:53431, leader=true}
[junit4:junit4]   2> 1571495 T3914 C8696 P53431 oasc.SolrCore.execute [nodes_used_collection_shard2_replica1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1 
[junit4:junit4]   2> 1571509 T3842 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1571522 T3868 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1571533 T3817 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 1571535 T3817 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:53431__awholynewcollection_0_shard1_replica1",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"awholynewcollection_0_shard1_replica1",
[junit4:junit4]   2> 	  "collection":"awholynewcollection_0",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:53431_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:53431"}
[junit4:junit4]   2> 1571540 T3812 C8696 P53431 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d2d28b lockFactory=org.apache.lucene.store.NativeFSLockFactory@4898a9),segFN=segments_1,generation=1}
[junit4:junit4]   2> 1571541 T3812 C8696 P53431 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 1571545 T3868 oass.IndexSchema.readSchema [awholynewcollection_1_shard2_replica2] Schema name=test
[junit4:junit4]   2> 1571554 T3812 C8696 P53431 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d2d28b lockFactory=org.apache.lucene.store.NativeFSLockFactory@4898a9),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d2d28b lockFactory=org.apache.lucene.store.NativeFSLockFactory@4898a9),segFN=segments_2,generation=2}
[junit4:junit4]   2> 1571554 T3812 C8696 P53431 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 1571554 T3817 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:53443__awholynewcollection_0_shard1_replica2",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"awholynewcollection_0_shard1_replica2",
[junit4:junit4]   2> 	  "collection":"awholynewcollection_0",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:53443_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:53443"}
[junit4:junit4]   2> 1571556 T3812 C8696 P53431 oass.SolrIndexSearcher.<init> Opening Searcher@4cf566 realtime
[junit4:junit4]   2> 1571558 T3812 C8696 P53431 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 1571558 T3812 C8696 P53431 oasup.LogUpdateProcessor.finish [nodes_used_collection_shard2_replica1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 129
[junit4:junit4]   2> ASYNC  NEW_CORE C8697 name=nodes_used_collection_shard2_replica2 org.apache.solr.core.SolrCore@19148a6 url=http://127.0.0.1:53452/nodes_used_collection_shard2_replica2 node=127.0.0.1:53452_ C8697_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica2 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica2, collection=nodes_used_collection, node_name=127.0.0.1:53452_, base_url=http://127.0.0.1:53452}
[junit4:junit4]   2> 1571559 T3889 C8697 P53452 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 1571559 T3889 C8697 P53452 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 1571561 T3812 C8696 P53431 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 1571561 T3812 C8696 P53431 oasc.SolrCore.execute [nodes_used_collection_shard2_replica1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 1571563 T3889 C8697 P53452 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 1571563 T3889 C8697 P53452 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 1571563 T3889 C8697 P53452 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 1571566 T3812 C8696 P53431 oasc.SolrCore.execute [nodes_used_collection_shard2_replica1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0 
[junit4:junit4]   2> 1571566 T3889 C8697 P53452 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 1571569 T3889 C8697 P53452 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1370387431519\nodes_used_collection_shard2_replica2\data\index.20130604161043195
[junit4:junit4]   2> 1571571 T3889 C8697 P53452 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@1803d5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ffc59a) fullCopy=false
[junit4:junit4]   2> 1571576 T3812 C8696 P53431 oasc.SolrCore.execute [nodes_used_collection_shard2_replica1] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 1571579 T3889 C8697 P53452 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 1571607 T3809 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1571655 T3817 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:53470__nodes_used_collection_shard1_replica2",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard1_replica2",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:53470_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:53470"}
[junit4:junit4]   2> 1571664 T3808 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1571670 T3825 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> ASYNC  NEW_CORE C8698 name=nodes_used_collection_shard2_replica2 org.apache.solr.core.SolrCore@19148a6 url=http://127.0.0.1:53452/nodes_used_collection_shard2_replica2 node=127.0.0.1:53452_ C8698_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica2 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica2, collection=nodes_used_collection, node_name=127.0.0.1:53452_, base_url=http://127.0.0.1:53452}
[junit4:junit4]   2> 1571701 T3889 C8698 P53452 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 1571701 T3889 C8698 P53452 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=nodes_used_collection_shard2_replica2
[junit4:junit4]   2> 1571705 T3872 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1571708 T3871 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1571755 T3827 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1571784 T3843 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1571836 T3827 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1571839 T3809 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> ASYNC  NEW_CORE C8699 name=nodes_used_collection_shard2_replica2 org.apache.solr.core.SolrCore@19148a6 url=http://127.0.0.1:53452/nodes_used_collection_shard2_replica2 node=127.0.0.1:53452_ C8699_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica2 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica2, collection=nodes_used_collection, node_name=127.0.0.1:53452_, base_url=http://127.0.0.1:53452}
[junit4:junit4]   2> 1571839 T3889 C8699 P53452 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@19057f1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@116c37),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@19057f1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@116c37),segFN=segments_2,generation=2}
[junit4:junit4]   2> 1571840 T3889 C8699 P53452 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 1571852 T3855 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1571876 T3858 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> ASYNC  NEW_CORE C8700 name=nodes_used_collection_shard2_replica2 org.apache.solr.core.SolrCore@19148a6 url=http://127.0.0.1:53452/nodes_used_collection_shard2_replica2 node=127.0.0.1:53452_ C8700_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica2 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica2, collection=nodes_used_collection, node_name=127.0.0.1:53452_, base_url=http://127.0.0.1:53452}
[junit4:junit4]   2> 1571912 T3889 C8700 P53452 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 1571912 T3889 C8700 P53452 oass.SolrIndexSearcher.<init> Opening Searcher@9231c9 main
[junit4:junit4]   2> 1571915 T3827 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1571921 T3827 oass.IndexSchema.readSchema [awholynewcollection_1_shard5_replica1] Schema name=test
[junit4:junit4]   2> 1571915 T3884 oasc.SolrCore.registerSearcher [nodes_used_collection_shard2_replica2] Registered new searcher Searcher@9231c9 main{StandardDirectoryReader(segments_2:1:n

[...truncated too long message...]

:junit4]   1>        /solr/collections/awholynewcollection_1/leader_elect/shard3/election/89809710504345608-127.0.0.1:53452__awholynewcollection_1_shard3_replica3-n_0000000003 (0)
[junit4:junit4]   1>        /solr/collections/awholynewcollection_1/leader_elect/shard3/election/89809710504345608-127.0.0.1:53452__awholynewcollection_1_shard3_replica3-n_0000000002 (0)
[junit4:junit4]   1>        /solr/collections/awholynewcollection_1/leader_elect/shard3/election/89809710504345603-127.0.0.1:53431__awholynewcollection_1_shard3_replica1-n_0000000005 (0)
[junit4:junit4]   1>     /solr/collections/awholynewcollection_1/leaders (5)
[junit4:junit4]   1>      /solr/collections/awholynewcollection_1/leaders/shard1 (0)
[junit4:junit4]   1>      DATA:
[junit4:junit4]   1>          {
[junit4:junit4]   1>            "core":"awholynewcollection_1_shard1_replica3",
[junit4:junit4]   1>            "node_name":"127.0.0.1:53443_",
[junit4:junit4]   1>            "base_url":"http://127.0.0.1:53443"}
[junit4:junit4]   1>      /solr/collections/awholynewcollection_1/leaders/shard4 (0)
[junit4:junit4]   1>      DATA:
[junit4:junit4]   1>          {
[junit4:junit4]   1>            "core":"awholynewcollection_1_shard4_replica3",
[junit4:junit4]   1>            "node_name":"127.0.0.1:53431_",
[junit4:junit4]   1>            "base_url":"http://127.0.0.1:53431"}
[junit4:junit4]   1>      /solr/collections/awholynewcollection_1/leaders/shard5 (0)
[junit4:junit4]   1>      DATA:
[junit4:junit4]   1>          {
[junit4:junit4]   1>            "core":"awholynewcollection_1_shard5_replica1",
[junit4:junit4]   1>            "node_name":"127.0.0.1:53443_",
[junit4:junit4]   1>            "base_url":"http://127.0.0.1:53443"}
[junit4:junit4]   1>      /solr/collections/awholynewcollection_1/leaders/shard2 (0)
[junit4:junit4]   1>      DATA:
[junit4:junit4]   1>          {
[junit4:junit4]   1>            "core":"awholynewcollection_1_shard2_replica3",
[junit4:junit4]   1>            "node_name":"127.0.0.1:53461_",
[junit4:junit4]   1>            "base_url":"http://127.0.0.1:53461"}
[junit4:junit4]   1>      /solr/collections/awholynewcollection_1/leaders/shard3 (0)
[junit4:junit4]   1>      DATA:
[junit4:junit4]   1>          {
[junit4:junit4]   1>            "core":"awholynewcollection_1_shard3_replica2",
[junit4:junit4]   1>            "node_name":"127.0.0.1:53443_",
[junit4:junit4]   1>            "base_url":"http://127.0.0.1:53443"}
[junit4:junit4]   1>   /solr/overseer_elect (2)
[junit4:junit4]   1>    /solr/overseer_elect/election (5)
[junit4:junit4]   1>     /solr/overseer_elect/election/89809710504345612-127.0.0.1:53470_-n_0000000004 (0)
[junit4:junit4]   1>     /solr/overseer_elect/election/89809710504345610-127.0.0.1:53461_-n_0000000003 (0)
[junit4:junit4]   1>     /solr/overseer_elect/election/89809710504345608-127.0.0.1:53452_-n_0000000002 (0)
[junit4:junit4]   1>     /solr/overseer_elect/election/89809710504345606-127.0.0.1:53443_-n_0000000001 (0)
[junit4:junit4]   1>     /solr/overseer_elect/election/89809710504345603-127.0.0.1:53431_-n_0000000000 (0)
[junit4:junit4]   1>    /solr/overseer_elect/leader (0)
[junit4:junit4]   1>    DATA:
[junit4:junit4]   1>        {"id":"89809710504345603-127.0.0.1:53431_-n_0000000000"}
[junit4:junit4]   1>  /zookeeper (1)
[junit4:junit4]   1>  DATA:
[junit4:junit4]   1>      
[junit4:junit4]   1> 
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=CollectionsAPIDistributedZkTest -Dtests.method=testDistribSearch -Dtests.seed=28BD50ED2F50C07D -Dtests.slow=true -Dtests.locale=de_DE -Dtests.timezone=America/Hermosillo -Dtests.file.encoding=UTF-8
[junit4:junit4] FAILURE  275s | CollectionsAPIDistributedZkTest.testDistribSearch <<<
[junit4:junit4]    > Throwable #1: java.lang.AssertionError: There are still nodes recoverying - waited for 230 seconds
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([28BD50ED2F50C07D:A95BDEF5580FA041]:0)
[junit4:junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:173)
[junit4:junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131)
[junit4:junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:126)
[junit4:junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:512)
[junit4:junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:146)
[junit4:junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 1831423 T3794 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> NOTE: test params are: codec=Lucene41, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=de_DE, timezone=America/Hermosillo
[junit4:junit4]   2> NOTE: Windows 7 6.1 x86/Oracle Corporation 1.7.0_21 (32-bit)/cpus=2,threads=2,free=20534120,total=194768896
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestSolr4Spatial, DisMaxRequestHandlerTest, TestJmxMonitoredMap, TestCharFilters, TestRealTimeGet, TestRandomFaceting, TestSort, TestIndexSearcher, QueryResultKeyTest, TestGroupingSearch, TestCollationField, SpellCheckCollatorTest, CoreContainerCoreInitFailuresTest, DefaultValueUpdateProcessorTest, FileBasedSpellCheckerTest, HardAutoCommitTest, TestDistributedSearch, TestDocSet, SolrInfoMBeanTest, BasicDistributedZkTest, TestFieldTypeCollectionResource, TestFieldResource, StatsComponentTest, UUIDFieldTest, TestLazyCores, TestReversedWildcardFilterFactory, TestIBSimilarityFactory, PrimitiveFieldTypeTest, TestExtendedDismaxParser, TestOmitPositions, AlternateDirectoryTest, TimeZoneUtilsTest, PluginInfoTest, PingRequestHandlerTest, RequestHandlersTest, DistanceFunctionTest, XmlUpdateRequestHandlerTest, ResourceLoaderTest, TestStressLucene, TestAtomicUpdateErrorCases, TestStressVersions, TestSystemIdResolver, PolyFieldTest, TestSchemaSimilarityResource, SolrIndexSplitterTest, CurrencyFieldOpenExchangeTest, ReturnFieldsTest, EchoParamsTest, LeaderElectionTest, MultiTermTest, TestQuerySenderNoQuery, BadCopyFieldTest, SystemInfoHandlerTest, SolrTestCaseJ4Test, TestSolrQueryParser, TestMaxScoreQueryParser, TestRecovery, TestFieldTypeResource, TestCoreDiscovery, RecoveryZkTest, TestMultiCoreConfBootstrap, SuggesterWFSTTest, TestSolrXml, PreAnalyzedFieldTest, OutputWriterTest, TestPerFieldSimilarity, TestFastLRUCache, OpenExchangeRatesOrgProviderTest, TestSearchPerf, TestPHPSerializedResponseWriter, TestDocumentBuilder, TestPhraseSuggestions, IndexReaderFactoryTest, DocValuesMultiTest, SolrCoreTest, StatelessScriptUpdateProcessorFactoryTest, HighlighterConfigTest, TestCoreContainer, LeaderElectionIntegrationTest, TestReloadAndDeleteDocs, SortByFunctionTest, ShardRoutingTest, TestSolrDeletionPolicy1, OpenCloseCoreStressTest, TestUpdate, TestBinaryResponseWriter, IndexBasedSpellCheckerTest, OverseerCollectionProcessorTest, TestSweetSpotSimilarityFactory, TestCSVLoader, IndexSchemaTest, LoggingHandlerTest, TestCloudManagedSchemaAddField, TestReload, ConvertedLegacyTest, TestFunctionQuery, BasicDistributedZk2Test, AutoCommitTest, URLClassifyProcessorTest, DistributedSpellCheckComponentTest, TestRandomDVFaceting, DistributedTermsComponentTest, UnloadDistributedZkTest, SpatialFilterTest, TestFieldCollectionResource, TestPropInjectDefaults, JSONWriterTest, WordBreakSolrSpellCheckerTest, SchemaVersionSpecificBehaviorTest, TestSolrQueryParserDefaultOperatorResource, QueryElevationComponentTest, TestAddFieldRealTimeGet, TestMergePolicyConfig, TestDFRSimilarityFactory, TestSerializedLuceneMatchVersion, FieldMutatingUpdateProcessorTest, ShardRoutingCustomTest, DateMathParserTest, TestBinaryField, DirectSolrConnectionTest, SuggesterTest, BinaryUpdateRequestHandlerTest, IndexSchemaRuntimeFieldTest, TestLRUCache, DebugComponentTest, ChaosMonkeyNothingIsSafeTest, ChaosMonkeySafeLeaderTest, DocumentBuilderTest, TestPartialUpdateDeduplication, DirectUpdateHandlerTest, TestHashPartitioner, PeerSyncTest, StandardRequestHandlerTest, TestDynamicFieldResource, PreAnalyzedUpdateProcessorTest, ClusterStateUpdateTest, TermsComponentTest, TestLMJelinekMercerSimilarityFactory, TestRemoteStreaming, DocValuesTest, TestQueryUtils, TestSolrDeletionPolicy2, TestLMDirichletSimilarityFactory, CollectionsAPIDistributedZkTest]
[junit4:junit4] Completed in 274.74s, 1 test, 1 failure <<< FAILURES!

[...truncated 463 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:380: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:360: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:39: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build.xml:181: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\common-build.xml:437: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1240: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:884: There were test failures: 296 suites, 1235 tests, 1 failure, 18 ignored (12 assumptions)

Total time: 70 minutes 23 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.7.0_21 -server -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure