You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2013/05/28 08:24:35 UTC

[JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.7.0_21) - Build # 2870 - Still Failing!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/2870/
Java: 64bit/jdk1.7.0_21 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
REGRESSION:  org.apache.solr.cloud.ChaosMonkeyShardSplitTest.testDistribSearch

Error Message:
Server at http://127.0.0.1:57232 returned non ok status:500, message:Server Error

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:57232 returned non ok status:500, message:Server Error
	at __randomizedtesting.SeedInfo.seed([7392E3A33341AD2:86DFA022446B7AEE]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
	at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:208)
	at org.apache.solr.cloud.ChaosMonkeyShardSplitTest.doTest(ChaosMonkeyShardSplitTest.java:136)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)




Build Log:
[...truncated 9254 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ChaosMonkeyShardSplitTest
[junit4:junit4]   2> 518888 T1519 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4]   2> 518902 T1519 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\solrtest-ChaosMonkeyShardSplitTest-1369720268375
[junit4:junit4]   2> 518907 T1519 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 518910 T1520 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 519020 T1519 oasc.ZkTestServer.run start zk server on port:57213
[junit4:junit4]   2> 519024 T1519 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 519042 T1526 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4c543c91 name:ZooKeeperConnection Watcher:127.0.0.1:57213 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 519042 T1519 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 519043 T1519 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 519057 T1519 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 519062 T1528 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@68ea148d name:ZooKeeperConnection Watcher:127.0.0.1:57213/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 519063 T1519 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 519063 T1519 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 519073 T1519 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 519080 T1519 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 519086 T1519 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 519094 T1519 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 519096 T1519 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 519109 T1519 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\schema15.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 519110 T1519 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 519118 T1519 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 519129 T1519 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 519138 T1519 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 519140 T1519 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 519147 T1519 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 519147 T1519 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 519155 T1519 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 519156 T1519 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 519163 T1519 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 519164 T1519 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 519172 T1519 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 519173 T1519 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 519181 T1519 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 519182 T1519 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 519196 T1521 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13ee9b0a6640001, likely client has closed socket
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 
[junit4:junit4]   2> 519871 T1519 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 519882 T1519 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:?????
[junit4:junit4]   2> 519883 T1519 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 519884 T1519 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 519885 T1519 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369720268661
[junit4:junit4]   2> 519888 T1519 oasc.CoreContainer$Initializer.initialize looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369720268661\solr.xml
[junit4:junit4]   2> 519888 T1519 oasc.CoreContainer.<init> New CoreContainer 1789314111
[junit4:junit4]   2> 519889 T1519 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369720268661\'
[junit4:junit4]   2> 519891 T1519 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369720268661\'
[junit4:junit4]   2> 520055 T1519 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 520056 T1519 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 520056 T1519 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 520057 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 520057 T1519 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 520057 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 520058 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 520058 T1519 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 520058 T1519 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 520059 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 520069 T1519 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 520070 T1519 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:57213/solr
[junit4:junit4]   2> 520071 T1519 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 520074 T1519 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 520113 T1539 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3957761d name:ZooKeeperConnection Watcher:127.0.0.1:57213 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 520114 T1519 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 520161 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 520161 T1521 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13ee9b0a6640002, likely client has closed socket
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 
[junit4:junit4]   2> 520172 T1519 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 520204 T1541 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@578dc449 name:ZooKeeperConnection Watcher:127.0.0.1:57213/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 520204 T1519 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 520209 T1519 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4]   2> 520222 T1519 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4]   2> 520230 T1519 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 520234 T1519 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57220_
[junit4:junit4]   2> 520240 T1519 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57220_
[junit4:junit4]   2> 520253 T1519 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 520276 T1519 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 520288 T1519 oasc.Overseer.start Overseer (id=89765987509075971-127.0.0.1:57220_-n_0000000000) starting
[junit4:junit4]   2> 520304 T1519 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4]   2> 520319 T1543 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 520320 T1519 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 520328 T1519 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4]   2> 520334 T1519 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 520341 T1542 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 520357 T1544 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369720268661\collection1
[junit4:junit4]   2> 520357 T1544 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 520360 T1544 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 520360 T1544 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 520364 T1544 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369720268661\collection1\'
[junit4:junit4]   2> 520368 T1544 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369720268661/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 520369 T1544 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369720268661/collection1/lib/README' to classloader
[junit4:junit4]   2> 520461 T1544 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 520564 T1544 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 520569 T1544 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 520579 T1544 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 521449 T1544 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 521457 T1544 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 521460 T1544 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 521465 T1544 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 521519 T1544 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 521520 T1544 oasc.SolrCore.<init> [collection1] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369720268661\collection1\, dataDir=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/control/data\
[junit4:junit4]   2> 521520 T1544 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7cbecd86
[junit4:junit4]   2> 521521 T1544 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 521522 T1544 oasc.CachingDirectoryFactory.get return new directory for .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/control/data\
[junit4:junit4]   2> 521522 T1544 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/control/data\index/
[junit4:junit4]   2> 521523 T1544 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371\control\data\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 521523 T1544 oasc.CachingDirectoryFactory.get return new directory for .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/control/data\index
[junit4:junit4]   2> 521528 T1544 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@134da6e9 lockFactory=org.apache.lucene.store.NativeFSLockFactory@481bf7c5),segFN=segments_1,generation=1}
[junit4:junit4]   2> 521528 T1544 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 521535 T1544 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 521536 T1544 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 521536 T1544 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 521537 T1544 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 521539 T1544 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 521539 T1544 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 521539 T1544 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 521541 T1544 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 521542 T1544 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 521558 T1544 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 521570 T1544 oass.SolrIndexSearcher.<init> Opening Searcher@589b8516 main
[junit4:junit4]   2> 521571 T1544 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371\control\data\tlog
[junit4:junit4]   2> 521573 T1544 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 521573 T1544 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 521581 T1545 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@589b8516 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 521582 T1544 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 521584 T1544 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 521997 T1542 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 521999 T1542 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"1",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57220_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57220"}
[junit4:junit4]   2> 521999 T1542 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 521999 T1542 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4]   2> 522012 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 522691 T1544 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 522691 T1544 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57220 collection:control_collection shard:shard1
[junit4:junit4]   2> 522694 T1544 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 522723 T1544 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 522733 T1544 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 522733 T1544 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 522734 T1544 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:57220/collection1/
[junit4:junit4]   2> 522734 T1544 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 522734 T1544 oasc.SyncStrategy.syncToMe http://127.0.0.1:57220/collection1/ has no replicas
[junit4:junit4]   2> 522734 T1544 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:57220/collection1/
[junit4:junit4]   2> 522735 T1544 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 523672 T1542 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 523699 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 523761 T1544 oasc.ZkController.register We are http://127.0.0.1:57220/collection1/ and leader is http://127.0.0.1:57220/collection1/
[junit4:junit4]   2> 523761 T1544 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57220
[junit4:junit4]   2> 523761 T1544 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 523761 T1544 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 523762 T1544 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 523766 T1544 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 523768 T1519 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
[junit4:junit4]   2> 523769 T1519 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 523771 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 523785 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 523790 T1519 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 523803 T1548 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@23078c14 name:ZooKeeperConnection Watcher:127.0.0.1:57213/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 523804 T1519 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 523806 T1519 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 523811 T1519 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 524424 T1519 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 524428 T1519 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:?????
[junit4:junit4]   2> 524430 T1519 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 524430 T1519 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 524431 T1519 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369720273283
[junit4:junit4]   2> 524431 T1519 oasc.CoreContainer$Initializer.initialize looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369720273283\solr.xml
[junit4:junit4]   2> 524433 T1519 oasc.CoreContainer.<init> New CoreContainer 1847290188
[junit4:junit4]   2> 524433 T1519 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369720273283\'
[junit4:junit4]   2> 524433 T1519 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369720273283\'
[junit4:junit4]   2> 524561 T1519 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 524561 T1519 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 524562 T1519 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 524562 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 524562 T1519 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 524562 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 524563 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 524563 T1519 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 524564 T1519 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 524564 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 524575 T1519 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 524576 T1519 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:57213/solr
[junit4:junit4]   2> 524577 T1519 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 524582 T1519 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 524606 T1559 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2247eb1b name:ZooKeeperConnection Watcher:127.0.0.1:57213 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 524607 T1519 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 524613 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 524625 T1519 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 524630 T1561 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@25ddbf05 name:ZooKeeperConnection Watcher:127.0.0.1:57213/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 524630 T1519 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 524640 T1519 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 525363 T1542 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 525365 T1542 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"1",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57220_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57220"}
[junit4:junit4]   2> 525382 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 525382 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 525382 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 525748 T1519 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57232_
[junit4:junit4]   2> 525752 T1519 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57232_
[junit4:junit4]   2> 525759 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 525759 T1548 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 525760 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 525760 T1561 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 525761 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 525764 T1541 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 525775 T1562 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369720273283\collection1
[junit4:junit4]   2> 525776 T1562 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 525777 T1562 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 525777 T1562 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 525779 T1562 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369720273283\collection1\'
[junit4:junit4]   2> 525783 T1562 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369720273283/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 525784 T1562 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369720273283/collection1/lib/README' to classloader
[junit4:junit4]   2> 525847 T1562 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 525923 T1562 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 525926 T1562 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 525932 T1562 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 526666 T1562 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 526670 T1562 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 526672 T1562 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 526685 T1562 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 526744 T1562 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 526744 T1562 oasc.SolrCore.<init> [collection1] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369720273283\collection1\, dataDir=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/jetty1\
[junit4:junit4]   2> 526745 T1562 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7cbecd86
[junit4:junit4]   2> 526745 T1562 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 526747 T1562 oasc.CachingDirectoryFactory.get return new directory for .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/jetty1\
[junit4:junit4]   2> 526747 T1562 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/jetty1\index/
[junit4:junit4]   2> 526747 T1562 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371\jetty1\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 526748 T1562 oasc.CachingDirectoryFactory.get return new directory for .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/jetty1\index
[junit4:junit4]   2> 526756 T1562 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1c33cd63 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3cfbef1a),segFN=segments_1,generation=1}
[junit4:junit4]   2> 526756 T1562 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 526763 T1562 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 526763 T1562 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 526764 T1562 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 526767 T1562 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 526768 T1562 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 526768 T1562 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 526769 T1562 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 526770 T1562 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 526770 T1562 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 526788 T1562 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 526807 T1562 oass.SolrIndexSearcher.<init> Opening Searcher@6d06a18d main
[junit4:junit4]   2> 526807 T1562 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371\jetty1\tlog
[junit4:junit4]   2> 526807 T1562 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 526807 T1562 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 526816 T1563 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6d06a18d main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 526820 T1562 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 526820 T1562 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 527044 T1542 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 527045 T1542 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"2",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57232_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57232"}
[junit4:junit4]   2> 527045 T1542 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4]   2> 527047 T1542 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4]   2> 527066 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 527066 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 527066 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 527931 T1562 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 527931 T1562 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57232 collection:collection1 shard:shard1
[junit4:junit4]   2> 527933 T1562 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 527994 T1562 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 528004 T1562 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 528004 T1562 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 528004 T1562 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:57232/collection1/
[junit4:junit4]   2> 528005 T1562 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 528005 T1562 oasc.SyncStrategy.syncToMe http://127.0.0.1:57232/collection1/ has no replicas
[junit4:junit4]   2> 528005 T1562 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:57232/collection1/
[junit4:junit4]   2> 528005 T1562 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 528715 T1542 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 528791 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 528795 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 528797 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 528857 T1562 oasc.ZkController.register We are http://127.0.0.1:57232/collection1/ and leader is http://127.0.0.1:57232/collection1/
[junit4:junit4]   2> 528857 T1562 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57232
[junit4:junit4]   2> 528858 T1562 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 528858 T1562 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 528858 T1562 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 528867 T1562 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 528868 T1519 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
[junit4:junit4]   2> 528868 T1519 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 528868 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 529520 T1519 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 529535 T1519 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:?????
[junit4:junit4]   2> 529536 T1519 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 529537 T1519 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 529539 T1519 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369720278350
[junit4:junit4]   2> 529539 T1519 oasc.CoreContainer$Initializer.initialize looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369720278350\solr.xml
[junit4:junit4]   2> 529539 T1519 oasc.CoreContainer.<init> New CoreContainer 2023301056
[junit4:junit4]   2> 529539 T1519 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369720278350\'
[junit4:junit4]   2> 529539 T1519 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369720278350\'
[junit4:junit4]   2> 529704 T1519 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 529705 T1519 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 529705 T1519 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 529706 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 529706 T1519 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 529708 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 529708 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 529709 T1519 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 529709 T1519 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 529709 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 529723 T1519 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 529723 T1519 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:57213/solr
[junit4:junit4]   2> 529724 T1519 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 529727 T1519 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 529746 T1575 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@103dd2c0 name:ZooKeeperConnection Watcher:127.0.0.1:57213 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 529747 T1519 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 529752 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 529752 T1521 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13ee9b0a6640007, likely client has closed socket
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 
[junit4:junit4]   2> 529759 T1519 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 529764 T1577 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@57dfc2cc name:ZooKeeperConnection Watcher:127.0.0.1:57213/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 529764 T1519 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 529775 T1519 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 530304 T1542 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 530306 T1542 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"2",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57232_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57232"}
[junit4:junit4]   2> 530316 T1577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 530317 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 530317 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 530316 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 530771 T1519 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57241_
[junit4:junit4]   2> 530774 T1519 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57241_
[junit4:junit4]   2> 530781 T1577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 530781 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 530783 T1561 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 530783 T1548 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 530784 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 530784 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 530790 T1541 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 530791 T1577 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 530805 T1578 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369720278350\collection1
[junit4:junit4]   2> 530805 T1578 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 530806 T1578 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 530806 T1578 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 530809 T1578 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369720278350\collection1\'
[junit4:junit4]   2> 530811 T1578 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369720278350/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 530812 T1578 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369720278350/collection1/lib/README' to classloader
[junit4:junit4]   2> 530873 T1578 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 530943 T1578 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 530946 T1578 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 530951 T1578 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 531586 T1578 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 531589 T1578 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 531591 T1578 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 531597 T1578 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 531656 T1578 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 531657 T1578 oasc.SolrCore.<init> [collection1] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369720278350\collection1\, dataDir=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/jetty2\
[junit4:junit4]   2> 531657 T1578 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7cbecd86
[junit4:junit4]   2> 531657 T1578 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 531658 T1578 oasc.CachingDirectoryFactory.get return new directory for .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/jetty2\
[junit4:junit4]   2> 531658 T1578 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/jetty2\index/
[junit4:junit4]   2> 531658 T1578 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371\jetty2\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 531660 T1578 oasc.CachingDirectoryFactory.get return new directory for .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/jetty2\index
[junit4:junit4]   2> 531668 T1578 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@37637d96 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7bca555f),segFN=segments_1,generation=1}
[junit4:junit4]   2> 531668 T1578 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 531674 T1578 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 531674 T1578 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 531675 T1578 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 531676 T1578 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 531677 T1578 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 531677 T1578 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 531678 T1578 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 531679 T1578 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 531680 T1578 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 531696 T1578 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 531704 T1578 oass.SolrIndexSearcher.<init> Opening Searcher@685cc52b main
[junit4:junit4]   2> 531705 T1578 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371\jetty2\tlog
[junit4:junit4]   2> 531708 T1578 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 531708 T1578 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 531716 T1579 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@685cc52b main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 531720 T1578 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 531720 T1578 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 531814 T1542 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 531816 T1542 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"3",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57241_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57241"}
[junit4:junit4]   2> 531816 T1542 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 531816 T1542 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4]   2> 531828 T1577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 531828 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 531828 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 531828 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 532717 T1578 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 532717 T1578 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57241 collection:collection1 shard:shard2
[junit4:junit4]   2> 532719 T1578 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 532742 T1578 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 532753 T1578 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 532753 T1578 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 532753 T1578 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:57241/collection1/
[junit4:junit4]   2> 532754 T1578 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 532754 T1578 oasc.SyncStrategy.syncToMe http://127.0.0.1:57241/collection1/ has no replicas
[junit4:junit4]   2> 532754 T1578 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:57241/collection1/
[junit4:junit4]   2> 532754 T1578 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 533326 T1542 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 533354 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 533354 T1577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 533355 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 533355 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 533376 T1578 oasc.ZkController.register We are http://127.0.0.1:57241/collection1/ and leader is http://127.0.0.1:57241/collection1/
[junit4:junit4]   2> 533376 T1578 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57241
[junit4:junit4]   2> 533377 T1578 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 533377 T1578 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 533377 T1578 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 533383 T1578 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 533386 T1519 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
[junit4:junit4]   2> 533386 T1519 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 533387 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 533897 T1519 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 533904 T1519 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:?????
[junit4:junit4]   2> 533905 T1519 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 533906 T1519 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 533906 T1519 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369720282871
[junit4:junit4]   2> 533906 T1519 oasc.CoreContainer$Initializer.initialize looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369720282871\solr.xml
[junit4:junit4]   2> 533906 T1519 oasc.CoreContainer.<init> New CoreContainer 1392117106
[junit4:junit4]   2> 533907 T1519 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369720282871\'
[junit4:junit4]   2> 533908 T1519 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369720282871\'
[junit4:junit4]   2> 534027 T1519 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 534027 T1519 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 534028 T1519 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 534028 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 534029 T1519 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 534029 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 534029 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 534029 T1519 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 534030 T1519 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 534031 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 534038 T1519 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 534039 T1519 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:57213/solr
[junit4:junit4]   2> 534039 T1519 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 534042 T1519 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 534064 T1591 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@67666eba name:ZooKeeperConnection Watcher:127.0.0.1:57213 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 534065 T1519 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 534072 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 534084 T1519 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 534092 T1593 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7c33a5b2 name:ZooKeeperConnection Watcher:127.0.0.1:57213/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 534093 T1519 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 534106 T1519 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 534858 T1542 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 534859 T1542 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"3",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57241_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57241"}
[junit4:junit4]   2> 534872 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 534873 T1593 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 534874 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 534873 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 534872 T1577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 535104 T1519 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57250_
[junit4:junit4]   2> 535108 T1519 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57250_
[junit4:junit4]   2> 535113 T1577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 535114 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 535115 T1561 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 535115 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 535116 T1548 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 535116 T1593 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 535117 T1593 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 535117 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 535120 T1577 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 535121 T1541 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 535131 T1594 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369720282871\collection1
[junit4:junit4]   2> 535131 T1594 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 535132 T1594 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 535132 T1594 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 535134 T1594 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369720282871\collection1\'
[junit4:junit4]   2> 535137 T1594 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369720282871/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 535138 T1594 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369720282871/collection1/lib/README' to classloader
[junit4:junit4]   2> 535206 T1594 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 535303 T1594 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 535306 T1594 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 535312 T1594 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 535986 T1594 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 535993 T1594 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 535996 T1594 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 536002 T1594 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 536041 T1594 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 536042 T1594 oasc.SolrCore.<init> [collection1] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369720282871\collection1\, dataDir=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/jetty3\
[junit4:junit4]   2> 536042 T1594 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7cbecd86
[junit4:junit4]   2> 536042 T1594 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 536042 T1594 oasc.CachingDirectoryFactory.get return new directory for .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/jetty3\
[junit4:junit4]   2> 536042 T1594 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/jetty3\index/
[junit4:junit4]   2> 536042 T1594 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371\jetty3\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 536042 T1594 oasc.CachingDirectoryFactory.get return new directory for .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/jetty3\index
[junit4:junit4]   2> 536049 T1594 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2094910d lockFactory=org.apache.lucene.store.NativeFSLockFactory@184cdd35),segFN=segments_1,generation=1}
[junit4:junit4]   2> 536049 T1594 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 536054 T1594 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 536054 T1594 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 536055 T1594 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 536056 T1594 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 536056 T1594 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 536057 T1594 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 536057 T1594 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 536058 T1594 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 536058 T1594 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 536069 T1594 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 536076 T1594 oass.SolrIndexSearcher.<init> Opening Searcher@6d7fcdcf main
[junit4:junit4]   2> 536077 T1594 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371\jetty3\tlog
[junit4:junit4]   2> 536079 T1594 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 536079 T1594 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 536085 T1595 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6d7fcdcf main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 536088 T1594 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 536088 T1594 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 536368 T1542 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 536369 T1542 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"4",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57250_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57250"}
[junit4:junit4]   2> 536369 T1542 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 536369 T1542 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4]   2> 536380 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 536380 T1593 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 536380 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 536381 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 536380 T1577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 537086 T1594 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 537086 T1594 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57250 collection:collection1 shard:shard1
[junit4:junit4]   2> 537094 T1594 oasc.ZkController.register We are http://127.0.0.1:57250/collection1/ and leader is http://127.0.0.1:57232/collection1/
[junit4:junit4]   2> 537094 T1594 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57250
[junit4:junit4]   2> 537094 T1594 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 537094 T1594 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C1682 name=collection1 org.apache.solr.core.SolrCore@1d041736 url=http://127.0.0.1:57250/collection1 node=127.0.0.1:57250_ C1682_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:57250_, base_url=http://127.0.0.1:57250}
[junit4:junit4]   2> 537096 T1596 C1682 P57250 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 537098 T1596 C1682 P57250 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 537098 T1594 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 537098 T1596 C1682 P57250 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 537099 T1596 C1682 P57250 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 537103 T1519 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
[junit4:junit4]   2> 537104 T1519 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 537104 T1596 C1682 P57250 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 537104 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 537123 T1554 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4]   2> 537633 T1519 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 537639 T1519 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:?????
[junit4:junit4]   2> 537641 T1519 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 537641 T1519 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 537641 T1519 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369720286589
[junit4:junit4]   2> 537641 T1519 oasc.CoreContainer$Initializer.initialize looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369720286589\solr.xml
[junit4:junit4]   2> 537642 T1519 oasc.CoreContainer.<init> New CoreContainer 1180036922
[junit4:junit4]   2> 537642 T1519 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369720286589\'
[junit4:junit4]   2> 537642 T1519 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369720286589\'
[junit4:junit4]   2> 537764 T1519 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 537764 T1519 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 537765 T1519 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 537765 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 537766 T1519 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 537766 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 537766 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 537766 T1519 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 537767 T1519 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 537767 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 537776 T1519 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 537776 T1519 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:57213/solr
[junit4:junit4]   2> 537777 T1519 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 537780 T1519 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 537798 T1608 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6872e6d6 name:ZooKeeperConnection Watcher:127.0.0.1:57213 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 537799 T1519 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 537804 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 537811 T1519 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 537816 T1610 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@31ba8bb2 name:ZooKeeperConnection Watcher:127.0.0.1:57213/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 537817 T1519 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 537825 T1519 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 537876 T1542 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 537877 T1542 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"4",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57250_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57250"}
[junit4:junit4]   2> 537887 T1610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 537888 T1577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 537888 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 537889 T1593 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 537888 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 537889 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 538113 T1554 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4]   2> 538113 T1554 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=4&state=recovering&nodeName=127.0.0.1:57250_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=990 
[junit4:junit4]   2> 538787 T1519 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57260_
[junit4:junit4]   2> 538791 T1519 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57260_
[junit4:junit4]   2> 538796 T1610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 538797 T1577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 538798 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 538799 T1548 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 538799 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 538800 T1561 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 538801 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 538801 T1593 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 538802 T1593 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 538805 T1577 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 538806 T1541 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 538806 T1610 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 538818 T1611 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369720286589\collection1
[junit4:junit4]   2> 538818 T1611 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 538819 T1611 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 538819 T1611 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 538823 T1611 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369720286589\collection1\'
[junit4:junit4]   2> 538825 T1611 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369720286589/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 538827 T1611 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369720286589/collection1/lib/README' to classloader
[junit4:junit4]   2> 538905 T1611 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 538987 T1611 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 538990 T1611 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 538998 T1611 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 539643 T1611 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 539647 T1611 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 539650 T1611 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 539655 T1611 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 539699 T1611 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 539700 T1611 oasc.SolrCore.<init> [collection1] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369720286589\collection1\, dataDir=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/jetty4\
[junit4:junit4]   2> 539700 T1611 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7cbecd86
[junit4:junit4]   2> 539700 T1611 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 539700 T1611 oasc.CachingDirectoryFactory.get return new directory for .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/jetty4\
[junit4:junit4]   2> 539701 T1611 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/jetty4\index/
[junit4:junit4]   2> 539701 T1611 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371\jetty4\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 539702 T1611 oasc.CachingDirectoryFactory.get return new directory for .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/jetty4\index
[junit4:junit4]   2> 539706 T1611 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4a1b37e7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@68c157b8),segFN=segments_1,generation=1}
[junit4:junit4]   2> 539706 T1611 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 539710 T1611 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 539710 T1611 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 539712 T1611 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 539713 T1611 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 539714 T1611 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 539714 T1611 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 539714 T1611 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 539714 T1611 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 539714 T1611 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 539725 T1611 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 539732 T1611 oass.SolrIndexSearcher.<init> Opening Searcher@52f54af0 main
[junit4:junit4]   2> 539732 T1611 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371\jetty4\tlog
[junit4:junit4]   2> 539734 T1611 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 539734 T1611 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 539738 T1612 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@52f54af0 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 539740 T1611 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 539741 T1611 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> ASYNC  NEW_CORE C1683 name=collection1 org.apache.solr.core.SolrCore@1d041736 url=http://127.0.0.1:57250/collection1 node=127.0.0.1:57250_ C1683_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:57250_, base_url=http://127.0.0.1:57250}
[junit4:junit4]   2> 539932 T1596 C1683 P57250 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:57232/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 539932 T1596 C1683 P57250 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:57250 START replicas=[http://127.0.0.1:57232/collection1/] nUpdates=100
[junit4:junit4]   2> 539933 T1596 C1683 P57250 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 539934 T1596 C1683 P57250 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 539934 T1596 C1683 P57250 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 539935 T1596 C1683 P57250 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 539935 T1596 C1683 P57250 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 539935 T1596 C1683 P57250 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:57232/collection1/. core=collection1
[junit4:junit4]   2> 539935 T1596 C1683 P57250 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C1684 name=collection1 org.apache.solr.core.SolrCore@4e55e523 url=http://127.0.0.1:57232/collection1 node=127.0.0.1:57232_ C1684_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:57232_, base_url=http://127.0.0.1:57232, leader=true}
[junit4:junit4]   2> 539946 T1555 C1684 P57232 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=2 
[junit4:junit4]   2> 539952 T1554 C1684 P57232 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 539956 T1554 C1684 P57232 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1c33cd63 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3cfbef1a),segFN=segments_1,generation=1}
[junit4:junit4]   2> 539957 T1554 C1684 P57232 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 539958 T1554 C1684 P57232 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1c33cd63 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3cfbef1a),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1c33cd63 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3cfbef1a),segFN=segments_2,generation=2}
[junit4:junit4]   2> 539959 T1554 C1684 P57232 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 539959 T1554 C1684 P57232 oass.SolrIndexSearcher.<init> Opening Searcher@6014496c realtime
[junit4:junit4]   2> 539959 T1554 C1684 P57232 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 539960 T1554 C1684 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4]   2> 539961 T1596 C1683 P57250 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 539961 T1596 C1683 P57250 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 539965 T1557 C1684 P57232 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 539966 T1557 C1684 P57232 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 539967 T1596 C1683 P57250 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 539967 T1596 C1683 P57250 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 539967 T1596 C1683 P57250 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 539970 T1551 C1684 P57232 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1 
[junit4:junit4]   2> 539972 T1596 C1683 P57250 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 539973 T1596 C1683 P57250 oasc.CachingDirectoryFactory.get return new directory for .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/jetty3\index.20130528112129441
[junit4:junit4]   2> 539973 T1596 C1683 P57250 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@37c47242 lockFactory=org.apache.lucene.store.NativeFSLockFactory@599a8dcd) fullCopy=false
[junit4:junit4]   2> 539976 T1553 C1684 P57232 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=1 
[junit4:junit4]   2> 539978 T1596 C1683 P57250 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 539980 T1596 C1683 P57250 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 539981 T1596 C1683 P57250 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 539983 T1596 C1683 P57250 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2094910d lockFactory=org.apache.lucene.store.NativeFSLockFactory@184cdd35),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2094910d lockFactory=org.apache.lucene.store.NativeFSLockFactory@184cdd35),segFN=segments_2,generation=2}
[junit4:junit4]   2> 539985 T1596 C1683 P57250 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 539985 T1596 C1683 P57250 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 539985 T1596 C1683 P57250 oass.SolrIndexSearcher.<init> Opening Searcher@e743f7c main
[junit4:junit4]   2> 539986 T1595 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@e743f7c main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 539988 T1596 C1683 P57250 oasc.CachingDirectoryFactory.closeCacheValue looking to close .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/jetty3\index.20130528112129441 [CachedDir<<refCount=0;path=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/jetty3\index.20130528112129441;done=true>>]
[junit4:junit4]   2> 539988 T1596 C1683 P57250 oasc.CachingDirectoryFactory.close Closing directory: .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/jetty3\index.20130528112129441
[junit4:junit4]   2> 539988 T1596 C1683 P57250 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/jetty3\index.20130528112129441
[junit4:junit4]   2> 539988 T1596 C1683 P57250 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 539988 T1596 C1683 P57250 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 539988 T1596 C1683 P57250 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 539988 T1596 C1683 P57250 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 539992 T1596 C1683 P57250 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 540634 T1542 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 540635 T1542 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"5",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57260_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57260"}
[junit4:junit4]   2> 540636 T1542 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 540636 T1542 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4]   2> 540647 T1542 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"4",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57250_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57250"}
[junit4:junit4]   2> 540658 T1577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 540658 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 540658 T1593 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 540659 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 540658 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 540658 T1610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 541529 T1611 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 541529 T1611 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57260 collection:collection1 shard:shard2
[junit4:junit4]   2> 541539 T1611 oasc.ZkController.register We are http://127.0.0.1:57260/collection1/ and leader is http://127.0.0.1:57241/collection1/
[junit4:junit4]   2> 541539 T1611 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57260
[junit4:junit4]   2> 541540 T1611 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 541540 T1611 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C1685 name=collection1 org.apache.solr.core.SolrCore@6eafe6e2 url=http://127.0.0.1:57260/collection1 node=127.0.0.1:57260_ C1685_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:57260_, base_url=http://127.0.0.1:57260}
[junit4:junit4]   2> 541541 T1615 C1685 P57260 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 541542 T1615 C1685 P57260 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 541542 T1615 C1685 P57260 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 541542 T1615 C1685 P57260 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 541543 T1611 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 541545 T1615 C1685 P57260 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 541547 T1519 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
[junit4:junit4]   2> 541547 T1519 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 541547 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 541555 T1572 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4]   2> 541555 T1519 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 541556 T1519 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4]   2> 541556 T1519 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4]   2> 541558 T1519 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 542010 T1542 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 542015 T1542 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"5",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57260_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57260"}
[junit4:junit4]   2> 542039 T1610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 542040 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 542040 T1577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 542041 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 542042 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 542041 T1593 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 542446 T1572 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4]   2> 542446 T1572 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=5&state=recovering&nodeName=127.0.0.1:57260_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=891 
[junit4:junit4]   2> 542451 T1519 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 543344 T1519 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C1685_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:57260_, base_url=http://127.0.0.1:57260}
[junit4:junit4]   2> 544229 T1615 C1685 P57260 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:57241/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 544229 T1615 C1685 P57260 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:57260 START replicas=[http://127.0.0.1:57241/collection1/] nUpdates=100
[junit4:junit4]   2> 544230 T1615 C1685 P57260 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 544230 T1615 C1685 P57260 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 544230 T1615 C1685 P57260 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 544231 T1615 C1685 P57260 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 544231 T1615 C1685 P57260 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 544231 T1615 C1685 P57260 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:57241/collection1/. core=collection1
[junit4:junit4]   2> 544231 T1615 C1685 P57260 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C1686 name=collection1 org.apache.solr.core.SolrCore@2a7228e1 url=http://127.0.0.1:57241/collection1 node=127.0.0.1:57241_ C1686_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:57241_, base_url=http://127.0.0.1:57241, leader=true}
[junit4:junit4]   2> 544249 T1573 C1686 P57241 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 544251 T1519 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 544256 T1572 C1686 P57241 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 544261 T1572 C1686 P57241 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@37637d96 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7bca555f),segFN=segments_1,generation=1}
[junit4:junit4]   2> 544262 T1572 C1686 P57241 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 544263 T1572 C1686 P57241 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@37637d96 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7bca555f),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@37637d96 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7bca555f),segFN=segments_2,generation=2}
[junit4:junit4]   2> 544263 T1572 C1686 P57241 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 544264 T1572 C1686 P57241 oass.SolrIndexSearcher.<init> Opening Searcher@2a466554 realtime
[junit4:junit4]   2> 544264 T1572 C1686 P57241 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 544265 T1572 C1686 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4]   2> ASYNC  NEW_CORE C1687 name=collection1 org.apache.solr.core.SolrCore@6eafe6e2 url=http://127.0.0.1:57260/collection1 node=127.0.0.1:57260_ C1687_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:57260_, base_url=http://127.0.0.1:57260}
[junit4:junit4]   2> 544267 T1615 C1687 P57260 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 544267 T1615 C1687 P57260 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 544270 T1569 C1686 P57241 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 544270 T1569 C1686 P57241 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 544271 T1615 C1687 P57260 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 544272 T1615 C1687 P57260 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 544272 T1615 C1687 P57260 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 544275 T1570 C1686 P57241 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1 
[junit4:junit4]   2> 544275 T1615 C1687 P57260 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 544277 T1615 C1687 P57260 oasc.CachingDirectoryFactory.get return new directory for .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/jetty4\index.20130528112133744
[junit4:junit4]   2> 544277 T1615 C1687 P57260 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@7a8e51 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5bc4a942) fullCopy=false
[junit4:junit4]   2> 544280 T1571 C1686 P57241 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=1 
[junit4:junit4]   2> 544282 T1615 C1687 P57260 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 544283 T1615 C1687 P57260 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 544283 T1615 C1687 P57260 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 544286 T1615 C1687 P57260 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4a1b37e7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@68c157b8),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4a1b37e7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@68c157b8),segFN=segments_2,generation=2}
[junit4:junit4]   2> 544287 T1615 C1687 P57260 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 544287 T1615 C1687 P57260 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 544288 T1615 C1687 P57260 oass.SolrIndexSearcher.<init> Opening Searcher@3e02f5e7 main
[junit4:junit4]   2> 544288 T1612 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3e02f5e7 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 544289 T1615 C1687 P57260 oasc.CachingDirectoryFactory.closeCacheValue looking to close .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/jetty4\index.20130528112133744 [CachedDir<<refCount=0;path=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/jetty4\index.20130528112133744;done=true>>]
[junit4:junit4]   2> 544289 T1615 C1687 P57260 oasc.CachingDirectoryFactory.close Closing directory: .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/jetty4\index.20130528112133744
[junit4:junit4]   2> 544289 T1615 C1687 P57260 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/jetty4\index.20130528112133744
[junit4:junit4]   2> 544289 T1615 C1687 P57260 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 544289 T1615 C1687 P57260 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 544289 T1615 C1687 P57260 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 544289 T1615 C1687 P57260 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 544295 T1615 C1687 P57260 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 544729 T1542 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 544730 T1542 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"5",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57260_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57260"}
[junit4:junit4]   2> 544740 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 544740 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 544741 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 544740 T1610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 544741 T1593 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 544740 T1577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 545146 T1519 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 545146 T1519 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4]   2> ASYNC  NEW_CORE C1688 name=collection1 org.apache.solr.core.SolrCore@1d707d7b url=http://127.0.0.1:57220/collection1 node=127.0.0.1:57220_ C1688_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:57220_, base_url=http://127.0.0.1:57220, leader=true}
[junit4:junit4]   2> 545155 T1534 C1688 P57220 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 545159 T1534 C1688 P57220 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@134da6e9 lockFactory=org.apache.lucene.store.NativeFSLockFactory@481bf7c5),segFN=segments_1,generation=1}
[junit4:junit4]   2> 545161 T1534 C1688 P57220 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 545162 T1534 C1688 P57220 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@134da6e9 lockFactory=org.apache.lucene.store.NativeFSLockFactory@481bf7c5),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@134da6e9 lockFactory=org.apache.lucene.store.NativeFSLockFactory@481bf7c5),segFN=segments_2,generation=2}
[junit4:junit4]   2> 545162 T1534 C1688 P57220 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 545163 T1534 C1688 P57220 oass.SolrIndexSearcher.<init> Opening Searcher@4206f010 main
[junit4:junit4]   2> 545163 T1534 C1688 P57220 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 545164 T1545 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4206f010 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 545165 T1534 C1688 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 11
[junit4:junit4]   2> 545172 T1568 C1686 P57241 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 545173 T1568 C1686 P57241 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@37637d96 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7bca555f),segFN=segments_2,generation=2}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@37637d96 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7bca555f),segFN=segments_3,generation=3}
[junit4:junit4]   2> 545174 T1568 C1686 P57241 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4]   2> 545174 T1568 C1686 P57241 oass.SolrIndexSearcher.<init> Opening Searcher@6fcd831a main
[junit4:junit4]   2> 545174 T1568 C1686 P57241 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 545176 T1579 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6fcd831a main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 545177 T1568 C1686 P57241 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:57232/collection1/, StdNode: http://127.0.0.1:57250/collection1/, StdNode: http://127.0.0.1:57260/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4]   2> ASYNC  NEW_CORE C1689 name=collection1 org.apache.solr.core.SolrCore@4e55e523 url=http://127.0.0.1:57232/collection1 node=127.0.0.1:57232_ C1689_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:57232_, base_url=http://127.0.0.1:57232, leader=true}
[junit4:junit4]   2> 545182 T1554 C1689 P57232 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 545183 T1554 C1689 P57232 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1c33cd63 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3cfbef1a),segFN=segments_2,generation=2}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1c33cd63 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3cfbef1a),segFN=segments_3,generation=3}
[junit4:junit4]   2> 545183 T1554 C1689 P57232 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4]   2> 545186 T1554 C1689 P57232 oass.SolrIndexSearcher.<init> Opening Searcher@43eaaa4b main
[junit4:junit4]   2> 545186 T1554 C1689 P57232 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2>  C1687_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:57260_, base_url=http://127.0.0.1:57260}
[junit4:junit4]   2> 545189 T1605 C1687 P57260 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 545192 T1563 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@43eaaa4b main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> ASYNC  NEW_CORE C1690 name=collection1 org.apache.solr.core.SolrCore@1d041736 url=http://127.0.0.1:57250/collection1 node=127.0.0.1:57250_ C1690_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:57250_, base_url=http://127.0.0.1:57250}
[junit4:junit4]   2> 545192 T1587 C1690 P57250 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 545193 T1554 C1689 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 11
[junit4:junit4]   2> 545193 T1605 C1687 P57260 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4a1b37e7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@68c157b8),segFN=segments_2,generation=2}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4a1b37e7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@68c157b8),segFN=segments_3,generation=3}
[junit4:junit4]   2> 545195 T1605 C1687 P57260 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4]   2> 545195 T1587 C1690 P57250 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2094910d lockFactory=org.apache.lucene.store.NativeFSLockFactory@184cdd35),segFN=segments_2,generation=2}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2094910d lockFactory=org.apache.lucene.store.NativeFSLockFactory@184cdd35),segFN=segments_3,generation=3}
[junit4:junit4]   2> 545199 T1587 C1690 P57250 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4]   2> 545199 T1605 C1687 P57260 oass.SolrIndexSearcher.<init> Opening Searcher@5b36d7e main
[junit4:junit4]   2> 545199 T1587 C1690 P57250 oass.SolrIndexSearcher.<init> Opening Searcher@6ccc97e8 main
[junit4:junit4]   2> 545200 T1587 C1690 P57250 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 545199 T1605 C1687 P57260 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 545200 T1612 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5b36d7e main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 545201 T1595 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6ccc97e8 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 545201 T1605 C1687 P57260 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 12
[junit4:junit4]   2> 545201 T1587 C1690 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4]   2> 545203 T1568 C1686 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 31
[junit4:junit4]   2> 545205 T1519 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 545208 T1557 C1689 P57232 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0 
[junit4:junit4]   2> 545213 T1588 C1690 P57250 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 545216 T1569 C1686 P57241 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 545220 T1606 C1687 P57260 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 547015 T1537 C1688 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1436255829601812480)} 0 4
[junit4:junit4]   2> 547029 T1604 C1687 P57260 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1436255829610201088&update.from=http://127.0.0.1:57241/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1436255829610201088)} 0 5
[junit4:junit4]   2> 547037 T1586 C1690 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1436255829616492544&update.from=http://127.0.0.1:57232/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1436255829616492544)} 0 2
[junit4:junit4]   2> 547039 T1555 C1689 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1436255829616492544)} 0 15
[junit4:junit4]   2> 547040 T1572 C1686 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1436255829610201088)} 0 21
[junit4:junit4]   2> 547045 T1533 C1688 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[0 (1436255829635366912)]} 0 2
[junit4:junit4]   2> 547059 T1587 C1690 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1436255829644804096)]} 0 3
[junit4:junit4]   2> 547061 T1554 C1689 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[0 (1436255829644804096)]} 0 8
[junit4:junit4]   2> 547061 T1568 C1686 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[0]} 0 12
[junit4:junit4]   2> 547066 T1534 C1688 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1 (1436255829658435584)]} 0 1
[junit4:junit4]   2> 547072 T1588 C1690 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1436255829661581312)]} 0 2
[junit4:junit4]   2> 547073 T1557 C1689 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1 (1436255829661581312)]} 0 5
[junit4:junit4]   2> 547075 T1536 C1688 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[2 (1436255829668921344)]} 0 0
[junit4:junit4]   2> 547087 T1605 C1687 P57260 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1436255829676261376)]} 0 2
[junit4:junit4]   2> 547088 T1569 C1686 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[2 (1436255829676261376)]} 0 7
[junit4:junit4]   2> 547089 T1551 C1689 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[2]} 0 10
[junit4:junit4]   2> 547094 T1532 C1688 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[3 (1436255829687795712)]} 0 1
[junit4:junit4]   2> 547104 T1606 C1687 P57260 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1436255829695135744)]} 0 1
[junit4:junit4]   2> 547105 T1570 C1686 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[3 (1436255829695135744)]} 0 5
[junit4:junit4]   2> 547107 T1553 C1689 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[3]} 0 10
[junit4:junit4]   2> 547111 T1535 C1688 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[4 (1436255829705621504)]} 0 1
[junit4:junit4]   2> 547121 T1589 C1690 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1436255829712961536)]} 0 0
[junit4:junit4]   2> 547122 T1555 C1689 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[4 (1436255829712961536)]} 0 5
[junit4:junit4]   2> 547123 T1571 C1686 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[4]} 0 8
[junit4:junit4]   2> 547128 T1537 C1688 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[5 (1436255829724495872)]} 0 1
[junit4:junit4]   2> 547139 T1601 C1687 P57260 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[5 (1436255829731835904)]} 0 1
[junit4:junit4]   2> 547141 T1573 C1686 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[5 (1436255829731835904)]} 0 6
[junit4:junit4]   2> 547142 T1556 C1689 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[5]} 0 10
[junit4:junit4]   2> 547145 T1533 C1688 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[6 (1436255829742321664)]} 0 0
[junit4:junit4]   2> 547154 T1602 C1687 P57260 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[6 (1436255829746515968)]} 0 1
[junit4:junit4]   2> 547155 T1572 C1686 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[6 (1436255829746515968)]} 0 6
[junit4:junit4]   2> 547158 T1534 C1688 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[7 (1436255829755953152)]} 0 1
[junit4:junit4]   2> 547168 T1603 C1687 P57260 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[7 (1436255829762244608)]} 0 0
[junit4:junit4]   2> 547169 T1568 C1686 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[7 (1436255829762244608)]} 0 6
[junit4:junit4]   2> 547170 T1554 C1689 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[7]} 0 9
[junit4:junit4]   2> 547173 T1536 C1688 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[8 (1436255829771681792)]} 0 1
[junit4:junit4]   2> 547183 T1584 C1690 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1436255829779021824)]} 0 1
[junit4:junit4]   2> 547184 T1557 C1689 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[8 (1436255829779021824)]} 0 5
[junit4:junit4]   2> 547185 T1569 C1686 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[8]} 0 10
[junit4:junit4]   2> 547189 T1532 C1688 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[9 (1436255829788459008)]} 0 1
[junit4:junit4]   2> 547199 T1604 C1687 P57260 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[9 (1436255829794750464)]} 0 2
[junit4:junit4]   2> 547200 T1570 C1686 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[9 (1436255829794750464)]} 0 6
[junit4:junit4]   2> 547201 T1551 C1689 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[9]} 0 10
[junit4:junit4]   2> 547204 T1535 C1688 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10 (1436255829804187648)]} 0 1
[junit4:junit4]   2> 547212 T1585 C1690 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1436255829808381952)]} 0 0
[junit4:junit4]   2> 547213 T1553 C1689 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10 (1436255829808381952)]} 0 6
[junit4:junit4]   2> 547218 T1537 C1688 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[11 (1436255829817819136)]} 0 1
[junit4:junit4]   2> 547227 T1586 C1690 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[11 (1436255829825159168)]} 0 1
[junit4:junit4]   2> 547229 T1555 C1689 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[11 (1436255829825159168)]} 0 6
[junit4:junit4]   2> 547229 T1571 C1686 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[11]} 0 8
[junit4:junit4]   2> 547233 T1533 C1688 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[12 (1436255829834596352)]} 0 0
[junit4:junit4]   2> 547243 T1587 C1690 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[12 (1436255829841936384)]} 0 1
[junit4:junit4]   2> 547245 T1556 C1689 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[12 (1436255829841936384)]} 0 6
[junit4:junit4]   2> 547245 T1573 C1686 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[12]} 0 8
[junit4:junit4]   2> 547249 T1534 C1688 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[13 (1436255829851373568)]} 0 1
[junit4:junit4]   2> 547259 T1588 C1690 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[13 (1436255829858713600)]} 0 1
[junit4:junit4]   2> 547261 T1554 C1689 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[13 (1436255829858713600)]} 0 5
[junit4:junit4]   2> 547262 T1572 C1686 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[13]} 0 10
[junit4:junit4]   2> 547266 T1536 C1688 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[14 (1436255829868150784)]} 0 1
[junit4:junit4]   2> 547276 T1589 C1690 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[14 (1436255829875490816)]} 0 1
[junit4:junit4]   2> 547277 T1557 C1689 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[14 (1436255829875490816)]} 0 5
[junit4:junit4]   2> ASYNC  NEW_CORE C1691 name=collection1 org.apache.solr.core.SolrCore@2a7228e1 url=http://127.0.0.1:57241/collection1 node=127.0.0.1:57241_ C1691_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:57241_, base_url=http://127.0.0.1:57241, leader=true}
[junit4:junit4]   2> 547287 T1568 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[14]} 0 18
[junit4:junit4]   2> ASYNC  NEW_CORE C1692 name=collection1 org.apache.solr.core.SolrCore@1d707d7b url=http://127.0.0.1:57220/collection1 node=127.0.0.1:57220_ C1692_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:57220_, base_url=http://127.0.0.1:57220, leader=true}
[junit4:junit4]   2> 547291 T1532 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[15 (1436255829894365184)]} 0 0
[junit4:junit4]   2> ASYNC  NEW_CORE C1693 name=collection1 org.apache.solr.core.SolrCore@1d041736 url=http://127.0.0.1:57250/collection1 node=127.0.0.1:57250_ C1693_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:57250_, base_url=http://127.0.0.1:57250}
[junit4:junit4]   2> 547302 T1584 C1693 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[15 (1436255829902753792)]} 0 1
[junit4:junit4]   2> ASYNC  NEW_CORE C1694 name=collection1 org.apache.solr.core.SolrCore@4e55e523 url=http://127.0.0.1:57232/collection1 node=127.0.0.1:57232_ C1694_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:57232_, base_url=http://127.0.0.1:57232, leader=true}
[junit4:junit4]   2> 547303 T1551 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[15 (1436255829902753792)]} 0 5
[junit4:junit4]   2> 547305 T1569 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[15]} 0 10
[junit4:junit4]   2> 547309 T1535 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[16 (1436255829914288128)]} 0 0
[junit4:junit4]   2> 547319 T1585 C1693 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[16 (1436255829921628160)]} 0 0
[junit4:junit4]   2> 547321 T1553 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[16 (1436255829921628160)]} 0 5
[junit4:junit4]   2> 547322 T1570 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[16]} 0 9
[junit4:junit4]   2> 547325 T1537 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[17 (1436255829931065344)]} 0 0
[junit4:junit4]   2> ASYNC  NEW_CORE C1695 name=collection1 org.apache.solr.core.SolrCore@6eafe6e2 url=http://127.0.0.1:57260/collection1 node=127.0.0.1:57260_ C1695_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:57260_, base_url=http://127.0.0.1:57260}
[junit4:junit4]   2> 547336 T1605 C1695 P57260 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[17 (1436255829937356800)]} 0 0
[junit4:junit4]   2> 547337 T1571 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[17 (1436255829937356800)]} 0 6
[junit4:junit4]   2> 547338 T1555 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[17]} 0 9
[junit4:junit4]   2> 547343 T1533 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[18 (1436255829948891136)]} 0 1
[junit4:junit4]   2> 547352 T1606 C1695 P57260 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[18 (1436255829956231168)]} 0 2
[junit4:junit4]   2> 547354 T1573 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[18 (1436255829956231168)]} 0 6
[junit4:junit4]   2> 547355 T1556 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[18]} 0 10
[junit4:junit4]   2> 547359 T1534 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[19 (1436255829965668352)]} 0 1
[junit4:junit4]   2> 547370 T1601 C1695 P57260 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[19 (1436255829973008384)]} 0 1
[junit4:junit4]   2> 547381 T1572 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[19 (1436255829973008384)]} 0 17
[junit4:junit4]   2> 547382 T1554 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[19]} 0 19
[junit4:junit4]   2> 547386 T1536 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[20 (1436255829995028480)]} 0 0
[junit4:junit4]   2> 547394 T1586 C1693 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[20 (1436255829999222784)]} 0 1
[junit4:junit4]   2> 547395 T1557 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[20 (1436255829999222784)]} 0 5
[junit4:junit4]   2> 547399 T1532 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[21 (1436255830008659968)]} 0 1
[junit4:junit4]   2> 547411 T1602 C1695 P57260 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[21 (1436255830017048576)]} 0 0
[junit4:junit4]   2> 547412 T1568 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[21 (1436255830017048576)]} 0 6
[junit4:junit4]   2> 547413 T1551 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[21]} 0 9
[junit4:junit4]   2> 547417 T1535 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[22 (1436255830027534336)]} 0 0
[junit4:junit4]   2> 547429 T1587 C1693 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[22 (1436255830036971520)]} 0 1
[junit4:junit4]   2> 547431 T1553 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[22 (1436255830036971520)]} 0 6
[junit4:junit4]   2> 547431 T1570 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[22]} 0 9
[junit4:junit4]   2> 547435 T1533 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[23 (1436255830046408704)]} 0 1
[junit4:junit4]   2> 547444 T1604 C1695 P57260 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[23 (1436255830051651584)]} 0 1
[junit4:junit4]   2> 547445 T1571 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[23 (1436255830051651584)]} 0 5
[junit4:junit4]   2> 547446 T1555 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[23]} 0 8
[junit4:junit4]   2> 547450 T1534 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[24 (1436255830062137344)]} 0 0
[junit4:junit4]   2> 547460 T1589 C1693 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[24 (1436255830069477376)]} 0 1
[junit4:junit4]   2> 547462 T1554 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[24 (1436255830069477376)]} 0 6
[junit4:junit4]   2> 547463 T1573 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[24]} 0 10
[junit4:junit4]   2> 547467 T1536 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[25 (1436255830079963136)]} 0 0
[junit4:junit4]   2> 547479 T1605 C1695 P57260 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[25 (1436255830088351744)]} 0 0
[junit4:junit4]   2> 547480 T1572 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[25 (1436255830088351744)]} 0 6
[junit4:junit4]   2> 547481 T1557 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[25]} 0 10
[junit4:junit4]   2> 547485 T1532 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[26 (1436255830098837504)]} 0 0
[junit4:junit4]   2> 547494 T1606 C1695 P57260 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[26 (1436255830104080384)]} 0 0
[junit4:junit4]   2> 547495 T1569 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[26 (1436255830104080384)]} 0 6
[junit4:junit4]   2> 547499 T1537 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[27 (1436255830113517568)]} 0 1
[junit4:junit4]   2> 547505 T1584 C1693 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[27 (1436255830116663296)]} 0 1
[junit4:junit4]   2> 547506 T1551 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[27 (1436255830116663296)]} 0 4
[junit4:junit4]   2> 547510 T1535 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[28 (1436255830125051904)]} 0 1
[junit4:junit4]   2> 547517 T1585 C1693 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[28 (1436255830129246208)]} 0 1
[junit4:junit4]   2> 547519 T1553 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[28 (1436255830129246208)]} 0 6
[junit4:junit4]   2> 547523 T1533 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[29 (1436255830138683392)]} 0 1
[junit4:junit4]   2> 547529 T1601 C1695 P57260 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[29 (1436255830141829120)]} 0 1
[junit4:junit4]   2> 547530 T1568 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[29 (1436255830141829120)]} 0 5
[junit4:junit4]   2> 547534 T1534 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[30 (1436255830150217728)]} 0 0
[junit4:junit4]   2> 547546 T1603 C1695 P57260 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[30 (1436255830158606336)]} 0 0
[junit4:junit4]   2> 547547 T1570 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[30 (1436255830158606336)]} 0 6
[junit4:junit4]   2> 547548 T1555 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[30]} 0 10
[junit4:junit4]   2> 547552 T1536 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[31 (1436255830169092096)]} 0 1
[junit4:junit4]   2> 547560 T1602 C1695 P57260 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[31 (1436255830174334976)]} 0 0
[junit4:junit4]   2> 547562 T1571 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[31 (1436255830174334976)]} 0 6
[junit4:junit4]   2> 547566 T1532 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[32 (1436255830183772160)]} 0 1
[junit4:junit4]   2> 547574 T1586 C1693 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[32 (1436255830187966464)]} 0 1
[junit4:junit4]   2> 547576 T1556 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[32 (1436255830187966464)]} 0 7
[junit4:junit4]   2> 547580 T1537 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[33 (1436255830198452224)]} 0 1
[junit4:junit4]   2> 547588 T1604 C1695 P57260 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[33 (1436255830203695104)]} 0 1
[junit4:junit4]   2> 547590 T1573 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[33 (1436255830203695104)]} 0 6
[junit4:junit4]   2> 547594 T1535 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[34 (1436255830213132288)]} 0 1
[junit4:junit4]   2> 547603 T1588 C1693 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[34 (1436255830218375168)]} 0 0
[junit4:junit4]   2> 547604 T1554 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[34 (1436255830218375168)]} 0 5
[junit4:junit4]   2> 547608 T1533 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[35 (1436255830227812352)]} 0 1
[junit4:junit4]   2> 547614 T1587 C1693 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[35 (1436255830232006656)]} 0 0
[junit4:junit4]   2> 547616 T1557 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[35 (1436255830232006656)]} 0 5
[junit4:junit4]   2> 547620 T1534 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[36 (1436255830240395264)]} 0 0
[junit4:junit4]   2> 547632 T1589 C1693 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[36 (1436255830249832448)]} 0 1
[junit4:junit4]   2> 547633 T1551 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[36 (1436255830249832448)]} 0 5
[junit4:junit4]   2> 547636 T1572 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[36]} 0 13
[junit4:junit4]   2> 547642 T1536 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[37 (1436255830263463936)]} 0 0
[junit4:junit4]   2> 547655 T1605 C1695 P57260 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[37 (1436255830272901120)]} 0 1
[junit4:junit4]   2> 547658 T1569 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[37 (1436255830272901120)]} 0 8
[junit4:junit4]   2> 547664 T1532 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[38 (1436255830286532608)]} 0 0
[junit4:junit4]   2> 547674 T1584 C1693 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[38 (1436255830292824064)]} 0 2
[junit4:junit4]   2> 547675 T1553 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[38 (1436255830292824064)]} 0 5
[junit4:junit4]   2> 547679 T1537 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[39 (1436255830302261248)]} 0 1
[junit4:junit4]   2> 547690 T1606 C1695 P57260 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[39 (1436255830309601280)]} 0 1
[junit4:junit4]   2> 547691 T1568 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[39 (1436255830309601280)]} 0 5
[junit4:junit4]   2> 547692 T1555 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[39]} 0 10
[junit4:junit4]   2> 547696 T1535 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[40 (1436255830320087040)]} 0 1
[junit4:junit4]   2> 547706 T1585 C1693 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[40 (1436255830326378496)]} 0 1
[junit4:junit4]   2> 547708 T1556 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[40 (1436255830326378496)]} 0 6
[junit4:junit4]   2> 547709 T1570 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[40]} 0 10
[junit4:junit4]   2> 547713 T1533 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[41 (1436255830337912832)]} 0 1
[junit4:junit4]   2> 547722 T1586 C1693 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[41 (1436255830343155712)]} 0 1
[junit4:junit4]   2> 547723 T1554 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[41 (1436255830343155712)]} 0 5
[junit4:junit4]   2> 547727 T1534 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[42 (1436255830352592896)]} 0 0
[junit4:junit4]   2> 547738 T1588 C1693 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[42 (1436255830360981504)]} 0 1
[junit4:junit4]   2> 547740 T1557 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[42 (1436255830360981504)]} 0 6
[junit4:junit4]   2> 547741 T1571 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[42]} 0 9
[junit4:junit4]   2> 547745 T1536 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[43 (1436255830371467264)]} 0 1
[junit4:junit4]   2> 547753 T1587 C1693 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[43 (1436255830375661568)]} 0 1
[junit4:junit4]   2> 547755 T1551 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[43 (1436255830375661568)]} 0 7
[junit4:junit4]   2> 547759 T1532 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[44 (1436255830386147328)]} 0 0
[junit4:junit4]   2> 547771 T1589 C1693 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[44 (1436255830395584512)]} 0 1
[junit4:junit4]   2> 547772 T1553 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[44 (1436255830395584512)]} 0 4
[junit4:junit4]   2> 547774 T1573 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[44]} 0 10
[junit4:junit4]   2> 547779 T1537 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[45 (1436255830406070272)]} 0 1
[junit4:junit4]   2> 547790 T1601 C1695 P57260 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[45 (1436255830414458880)]} 0 1
[junit4:junit4]   2> 547791 T1572 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[45 (1436255830414458880)]} 0 6
[junit4:junit4]   2> 547792 T1555 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[45]} 0 10
[junit4:junit4]   2> 547794 T1535 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[46 (1436255830421798912)]} 0 1
[junit4:junit4]   2> 547801 T1584 C1693 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[46 (1436255830427041792)]} 0 1
[junit4:junit4]   2> 547802 T1556 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[46 (1436255830427041792)]} 0 5
[junit4:junit4]   2> 547807 T1533 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[47 (1436255830436478976)]} 0 0
[junit4:junit4]   2> 547817 T1585 C1693 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[47 (1436255830443819008)]} 0 2
[junit4:junit4]   2> 547818 T1554 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[47 (1436255830443819008)]} 0 5
[junit4:junit4]   2> 547819 T1569 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[47]} 0 8
[junit4:junit4]   2> 547823 T1534 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[48 (1436255830453256192)]} 0 1
[junit4:junit4]   2> 547831 T1586 C1693 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[48 (1436255830456401920)]} 0 1
[junit4:junit4]   2> 547832 T1557 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[48 (1436255830456401920)]} 0 6
[junit4:junit4]   2> 547836 T1532 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[49 (1436255830465839104)]} 0 1
[junit4:junit4]   2> 547844 T1587 C1693 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[49 (1436255830470033408)]} 0 1
[junit4:junit4]   2> 547845 T1551 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[49 (1436255830470033408)]} 0 6
[junit4:junit4]   2> 547849 T1537 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[50 (1436255830480519168)]} 0 0
[junit4:junit4]   2> 547860 T1589 C1693 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[50 (1436255830488907776)]} 0 1
[junit4:junit4]   2> 547861 T1553 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[50 (1436255830488907776)]} 0 4
[junit4:junit4]   2> 547863 T1570 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[50]} 0 10
[junit4:junit4]   2> 547867 T1535 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[51 (1436255830498344960)]} 0 1
[junit4:junit4]   2> 547876 T1584 C1693 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[51 (1436255830503587840)]} 0 1
[junit4:junit4]   2> 547883 T1556 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[51 (1436255830503587840)]} 0 12
[junit4:junit4]   2> 547887 T1533 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[52 (1436255830520365056)]} 0 1
[junit4:junit4]   2> 547897 T1585 C1693 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[52 (1436255830526656512)]} 0 1
[junit4:junit4]   2> 547898 T1554 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[52 (1436255830526656512)]} 0 5
[junit4:junit4]   2> 547899 T1571 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[52]} 0 9
[junit4:junit4]   2> 547902 T1536 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[53 (1436255830536093696)]} 0 0
[junit4:junit4]   2> 547911 T1603 C1695 P57260 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[53 (1436255830542385152)]} 0 1
[junit4:junit4]   2> 547913 T1573 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[53 (1436255830542385152)]} 0 6
[junit4:junit4]   2> 547916 T1534 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[54 (1436255830550773760)]} 0 1
[junit4:junit4]   2> 547923 T1586 C1693 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[54 (1436255830554968064)]} 0 1
[junit4:junit4]   2> 547924 T1557 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[54 (1436255830554968064)]} 0 5
[junit4:junit4]   2> 547928 T1532 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[55 (1436255830563356672)]} 0 1
[junit4:junit4]   2> 547938 T1588 C1693 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[55 (1436255830570696704)]} 0 0
[junit4:junit4]   2> 547940 T1551 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[55 (1436255830570696704)]} 0 6
[junit4:junit4]   2> 547941 T1572 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[55]} 0 10
[junit4:junit4]   2> 547945 T1537 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[56 (1436255830581182464)]} 0 0
[junit4:junit4]   2> 547952 T1604 C1695 P57260 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[56 (1436255830584328192)]} 0 0
[junit4:junit4]   2> 547954 T1568 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[56 (1436255830584328192)]} 0 6
[junit4:junit4]   2> 547959 T1535 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[57 (1436255830594813952)]} 0 1
[junit4:junit4]   2> 547967 T1605 C1695 P57260 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[57 (1436255830600056832)]} 0 0
[junit4:junit4]   2> 547971 T1569 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[57 (1436255830600056832)]} 0 8
[junit4:junit4]   2> 547989 T1533 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[58 (1436255830626271232)]} 0 2
[junit4:junit4]   2> 548002 T1606 C1695 P57260 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[58 (1436255830635708416)]} 0 1
[junit4:junit4]   2> 548003 T1570 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[58 (1436255830635708416)]} 0 6
[junit4:junit4]   2> 548004 T1555 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[58]} 0 12
[junit4:junit4]   2> 548008 T1536 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[59 (1436255830647242752)]} 0 0
[junit4:junit4]   2> 548020 T1587 C1693 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[59 (1436255830654582784)]} 0 1
[junit4:junit4]   2> 548023 T1553 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[59 (1436255830654582784)]} 0 8
[junit4:junit4]   2> 548025 T1571 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[59]} 0 12
[junit4:junit4]   2> 548031 T1534 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[60 (1436255830670311424)]} 0 1
[junit4:junit4]   2> 548039 T1601 C1695 P57260 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[60 (1436255830676602880)]} 0 0
[junit4:junit4]   2> 548042 T1573 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[60 (1436255830676602880)]} 0 6
[junit4:junit4]   2> 548048 T1532 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[61 (1436255830688137216)]} 0 2
[junit4:junit4]   2> 548061 T1589 C1693 P57250 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57232/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[61 (1436255830698622976)]} 0 0
[junit4:junit4]   2> 548064 T1556 C1694 P57232 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57241/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[61 (1436255830698622976)]} 0 7
[junit4:junit4]   2> 548066 T1572 C1691 P57241 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[61]} 0 13
[junit4:junit4]   2> 548072 T1537 C1692 P57220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt

[...truncated too long message...]

	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":null,
[junit4:junit4]   2> 	  "core":"collection1_shard1_0_replica1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57250_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57250"}
[junit4:junit4]   2> 586221 T1610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 586225 T1519 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 57260
[junit4:junit4]   2> 586225 T1519 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1180036922
[junit4:junit4]   2> 587296 T1519 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 587296 T1519 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 587300 T1519 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@6eafe6e2
[junit4:junit4]   2> 587301 T1519 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=4,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=272,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4:junit4]   2> 587302 T1519 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 587302 T1519 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 587302 T1519 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 587308 T1519 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 587309 T1519 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4]   2> 587309 T1519 oasc.CachingDirectoryFactory.closeCacheValue looking to close .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/jetty4\index [CachedDir<<refCount=0;path=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/jetty4\index;done=false>>]
[junit4:junit4]   2> 587309 T1519 oasc.CachingDirectoryFactory.close Closing directory: .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/jetty4\index
[junit4:junit4]   2> 587310 T1519 oasc.CachingDirectoryFactory.closeCacheValue looking to close .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/jetty4\ [CachedDir<<refCount=0;path=.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/jetty4\;done=false>>]
[junit4:junit4]   2> 587310 T1519 oasc.CachingDirectoryFactory.close Closing directory: .\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371/jetty4\
[junit4:junit4]   2> 587313 T1670 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89765987509075980-127.0.0.1:57260_-n_0000000004) am no longer a leader.
[junit4:junit4]   2> 587316 T1610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 587330 T1519 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4:junit4]   2> !!!! WARNING: best effort to remove C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371\zookeeper\server1\data\version-2\log.1 FAILED !!!!!
[junit4:junit4]   2> !!!! WARNING: best effort to remove C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371\zookeeper\server1\data\version-2 FAILED !!!!!
[junit4:junit4]   2> !!!! WARNING: best effort to remove C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371\zookeeper\server1\data FAILED !!!!!
[junit4:junit4]   2> !!!! WARNING: best effort to remove C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371\zookeeper\server1 FAILED !!!!!
[junit4:junit4]   2> !!!! WARNING: best effort to remove C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371\zookeeper FAILED !!!!!
[junit4:junit4]   2> !!!! WARNING: best effort to remove C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369720268371 FAILED !!!!!
[junit4:junit4]   2> 587435 T1519 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4]   2> 587439 T1519 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:57213 57213
[junit4:junit4]   2> 588705 T1610 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 589843 T1610 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 589846 T1519 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 57220
[junit4:junit4]   2> 589863 T1519 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4]   2> 589864 T1519 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:57213 57213
[junit4:junit4]   1> client port:0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ChaosMonkeyShardSplitTest -Dtests.method=testDistribSearch -Dtests.seed=7392E3A33341AD2 -Dtests.slow=true -Dtests.locale=hi_IN -Dtests.timezone=Asia/Kolkata -Dtests.file.encoding=Cp1252
[junit4:junit4] ERROR   71.0s | ChaosMonkeyShardSplitTest.testDistribSearch <<<
[junit4:junit4]    > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:57232 returned non ok status:500, message:Server Error
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([7392E3A33341AD2:86DFA022446B7AEE]:0)
[junit4:junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
[junit4:junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4]    > 	at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:208)
[junit4:junit4]    > 	at org.apache.solr.cloud.ChaosMonkeyShardSplitTest.doTest(ChaosMonkeyShardSplitTest.java:136)
[junit4:junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 589875 T1519 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> NOTE: test params are: codec=Lucene41, sim=DefaultSimilarity, locale=hi_IN, timezone=Asia/Kolkata
[junit4:junit4]   2> NOTE: Windows 7 6.1 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=2,threads=2,free=18806728,total=113688576
[junit4:junit4]   2> NOTE: All tests run in this JVM: [CurrencyFieldXmlFileTest, OverseerCollectionProcessorTest, TestPerFieldSimilarity, TestPropInjectDefaults, FileUtilsTest, TestFastLRUCache, TestRandomDVFaceting, PingRequestHandlerTest, TestComponentsName, TestSolrDeletionPolicy1, TestReversedWildcardFilterFactory, TestUniqueKeyFieldResource, TestMaxScoreQueryParser, TestConfig, TestQueryTypes, TestValueSourceCache, TestSolrQueryParserResource, UniqFieldsUpdateProcessorFactoryTest, ZkCLITest, AliasIntegrationTest, TestFieldResource, JsonLoaderTest, LeaderElectionIntegrationTest, IndexSchemaRuntimeFieldTest, TestSuggestSpellingConverter, TestFieldTypeCollectionResource, ZkNodePropsTest, RecoveryZkTest, TestSolrCoreProperties, TestClassNameShortening, CacheHeaderTest, OverseerTest, TestWriterPerf, SpellPossibilityIteratorTest, PeerSyncTest, TestPhraseSuggestions, QueryResultKeyTest, SliceStateUpdateTest, TestJmxMonitoredMap, TestManagedSchemaFieldResource, DistributedQueryElevationComponentTest, UnloadDistributedZkTest, TestIndexingPerformance, QueryElevationComponentTest, HardAutoCommitTest, TestArbitraryIndexDir, SOLR749Test, CachingDirectoryFactoryTest, TestFastOutputStream, SolrTestCaseJ4Test, TestWordDelimiterFilterFactory, EchoParamsTest, RAMDirectoryFactoryTest, PrimUtilsTest, TestCodecSupport, ChaosMonkeyShardSplitTest]
[junit4:junit4] Completed in 71.03s, 1 test, 1 error <<< FAILURES!

[...truncated 747 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:380: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:360: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:39: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build.xml:181: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\common-build.xml:437: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1240: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:884: There were test failures: 295 suites, 1231 tests, 1 error, 18 ignored (12 assumptions)

Total time: 63 minutes 37 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0_21 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-trunk-Windows (32bit/jdk1.7.0_21) - Build # 2871 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/2871/
Java: 32bit/jdk1.7.0_21 -client -XX:+UseSerialGC

1 tests failed.
REGRESSION:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch

Error Message:
There are still nodes recoverying - waited for 230 seconds

Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 230 seconds
	at __randomizedtesting.SeedInfo.seed([C4186F94A266BA7E:45FEE18CD539DA42]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:173)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:126)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:512)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:146)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)




Build Log:
[...truncated 9567 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
[junit4:junit4]   2> 1542174 T4412 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4]   2> 1542178 T4412 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\solrtest-CollectionsAPIDistributedZkTest-1369747941387
[junit4:junit4]   2> 1542181 T4412 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 1542182 T4413 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 1542281 T4412 oasc.ZkTestServer.run start zk server on port:50126
[junit4:junit4]   2> 1542285 T4412 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1542305 T4419 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@98889c name:ZooKeeperConnection Watcher:127.0.0.1:50126 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1542306 T4412 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1542307 T4412 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 1542323 T4414 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13eeb56e8190000, likely client has closed socket
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 
[junit4:junit4]   2> 1542328 T4412 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1542336 T4421 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1394079 name:ZooKeeperConnection Watcher:127.0.0.1:50126/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1542337 T4412 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1542338 T4412 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 1542352 T4412 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 1542365 T4412 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 1542378 T4412 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 1542386 T4412 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 1542387 T4412 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 1542397 T4412 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 1542397 T4412 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 1542403 T4412 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 1542403 T4412 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 1542411 T4412 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 1542411 T4412 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 1542418 T4412 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 1542418 T4412 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 1542426 T4412 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 1542427 T4412 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 1542433 T4412 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 1542434 T4412 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 1542439 T4412 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 1542439 T4412 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 1542444 T4412 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 1542444 T4412 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 1542970 T4412 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 1542976 T4412 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50133
[junit4:junit4]   2> 1542977 T4412 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 1542978 T4412 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1542978 T4412 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369747941658
[junit4:junit4]   2> 1542978 T4412 oasc.CoreContainer$Initializer.initialize looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369747941658\solr.xml
[junit4:junit4]   2> 1542979 T4412 oasc.CoreContainer.<init> New CoreContainer 19625195
[junit4:junit4]   2> 1542979 T4412 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369747941658\'
[junit4:junit4]   2> 1542980 T4412 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369747941658\'
[junit4:junit4]   2> 1543087 T4412 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369747941658
[junit4:junit4]   2> 1543089 T4412 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369747941658\collection1
[junit4:junit4]   2> 1543091 T4412 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369747941658\collection1\conf
[junit4:junit4]   2> 1543364 T4412 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369747941658\collection1\conf\xslt
[junit4:junit4]   2> 1543374 T4412 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369747941658\collection1\lib
[junit4:junit4]   2> 1543379 T4412 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369747941658\collection1\lib\classes
[junit4:junit4]   2> 1543386 T4412 oasc.SolrCoreDiscoverer.addCore Discovered properties file C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369747941658\conf\core.properties, adding to cores
[junit4:junit4]   2> 1543410 T4412 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 1543410 T4412 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 1543411 T4412 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 1543411 T4412 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 1543412 T4412 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 1543413 T4412 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 1543413 T4412 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 1543414 T4412 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 1543415 T4412 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 1543415 T4412 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 1543426 T4412 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 1543426 T4412 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50126/solr
[junit4:junit4]   2> 1543427 T4412 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 1543429 T4412 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1543451 T4432 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1878f52 name:ZooKeeperConnection Watcher:127.0.0.1:50126 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1543452 T4412 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1543460 T4412 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 1543462 T4414 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13eeb56e8190002, likely client has closed socket
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 
[junit4:junit4]   2> 1543477 T4412 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1543484 T4434 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@60d442 name:ZooKeeperConnection Watcher:127.0.0.1:50126/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1543485 T4412 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1543491 T4412 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4]   2> 1543507 T4412 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4]   2> 1543519 T4412 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 1543527 T4412 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50133_
[junit4:junit4]   2> 1543531 T4412 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50133_
[junit4:junit4]   2> 1543547 T4412 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 1543562 T4412 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 1543568 T4412 oasc.Overseer.start Overseer (id=89767801087655939-127.0.0.1:50133_-n_0000000000) starting
[junit4:junit4]   2> 1543577 T4412 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4]   2> 1543585 T4436 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 1543587 T4412 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 1543591 T4412 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4]   2> 1543595 T4412 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1543599 T4435 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 1543602 T4412 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
[junit4:junit4]   2> 1543603 T4412 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 1543603 T4412 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 1543608 T4412 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 1543611 T4412 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1543616 T4439 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1adb3f0 name:ZooKeeperConnection Watcher:127.0.0.1:50126/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1543617 T4412 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1543618 T4412 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1543626 T4412 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 1544106 T4412 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 1544112 T4412 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50145
[junit4:junit4]   2> 1544112 T4412 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 1544112 T4412 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1544113 T4412 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369747942834
[junit4:junit4]   2> 1544113 T4412 oasc.CoreContainer$Initializer.initialize looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369747942834\solr.xml
[junit4:junit4]   2> 1544114 T4412 oasc.CoreContainer.<init> New CoreContainer 14205254
[junit4:junit4]   2> 1544114 T4412 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369747942834\'
[junit4:junit4]   2> 1544114 T4412 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369747942834\'
[junit4:junit4]   2> 1544216 T4412 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369747942834
[junit4:junit4]   2> 1544217 T4412 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369747942834\collection1
[junit4:junit4]   2> 1544221 T4412 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369747942834\collection1\conf
[junit4:junit4]   2> 1544488 T4412 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369747942834\collection1\conf\xslt
[junit4:junit4]   2> 1544497 T4412 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369747942834\collection1\lib
[junit4:junit4]   2> 1544501 T4412 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369747942834\collection1\lib\classes
[junit4:junit4]   2> 1544509 T4412 oasc.SolrCoreDiscoverer.addCore Discovered properties file C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369747942834\conf\core.properties, adding to cores
[junit4:junit4]   2> 1544528 T4412 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 1544529 T4412 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 1544529 T4412 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 1544529 T4412 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 1544529 T4412 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 1544530 T4412 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 1544530 T4412 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 1544530 T4412 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 1544531 T4412 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 1544531 T4412 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 1544539 T4412 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 1544540 T4412 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50126/solr
[junit4:junit4]   2> 1544540 T4412 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 1544543 T4412 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1544558 T4450 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e63c4a name:ZooKeeperConnection Watcher:127.0.0.1:50126 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1544558 T4412 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1544562 T4412 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 1544569 T4412 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1544571 T4452 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@77d8a5 name:ZooKeeperConnection Watcher:127.0.0.1:50126/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1544572 T4412 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1544580 T4412 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1545587 T4412 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50145_
[junit4:junit4]   2> 1545598 T4412 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50145_
[junit4:junit4]   2> 1545612 T4439 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 1545612 T4452 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 1545612 T4434 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 1545620 T4412 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
[junit4:junit4]   2> 1545620 T4412 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 1545620 T4412 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 1546117 T4412 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 1546123 T4412 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50154
[junit4:junit4]   2> 1546123 T4412 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 1546123 T4412 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1546124 T4412 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369747944833
[junit4:junit4]   2> 1546124 T4412 oasc.CoreContainer$Initializer.initialize looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369747944833\solr.xml
[junit4:junit4]   2> 1546125 T4412 oasc.CoreContainer.<init> New CoreContainer 18607049
[junit4:junit4]   2> 1546126 T4412 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369747944833\'
[junit4:junit4]   2> 1546127 T4412 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369747944833\'
[junit4:junit4]   2> 1546248 T4412 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369747944833
[junit4:junit4]   2> 1546250 T4412 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369747944833\collection1
[junit4:junit4]   2> 1546252 T4412 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369747944833\collection1\conf
[junit4:junit4]   2> 1546522 T4412 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369747944833\collection1\conf\xslt
[junit4:junit4]   2> 1546533 T4412 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369747944833\collection1\lib
[junit4:junit4]   2> 1546537 T4412 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369747944833\collection1\lib\classes
[junit4:junit4]   2> 1546545 T4412 oasc.SolrCoreDiscoverer.addCore Discovered properties file C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369747944833\conf\core.properties, adding to cores
[junit4:junit4]   2> 1546565 T4412 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 1546565 T4412 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 1546565 T4412 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 1546566 T4412 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 1546566 T4412 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 1546566 T4412 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 1546566 T4412 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 1546566 T4412 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 1546567 T4412 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 1546567 T4412 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 1546574 T4412 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 1546575 T4412 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50126/solr
[junit4:junit4]   2> 1546576 T4412 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 1546577 T4412 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1546596 T4464 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14f0d2c name:ZooKeeperConnection Watcher:127.0.0.1:50126 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1546596 T4412 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1546602 T4412 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 1546603 T4414 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13eeb56e8190007, likely client has closed socket
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 
[junit4:junit4]   2> 1546615 T4412 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1546619 T4466 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@bdeea name:ZooKeeperConnection Watcher:127.0.0.1:50126/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1546619 T4412 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1546628 T4412 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1547632 T4412 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50154_
[junit4:junit4]   2> 1547644 T4412 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50154_
[junit4:junit4]   2> 1547652 T4439 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 1547652 T4434 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 1547652 T4452 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 1547653 T4466 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 1547661 T4412 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
[junit4:junit4]   2> 1547661 T4412 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 1547661 T4412 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 1548084 T4412 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 1548088 T4412 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50163
[junit4:junit4]   2> 1548089 T4412 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 1548089 T4412 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1548089 T4412 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369747946874
[junit4:junit4]   2> 1548089 T4412 oasc.CoreContainer$Initializer.initialize looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369747946874\solr.xml
[junit4:junit4]   2> 1548091 T4412 oasc.CoreContainer.<init> New CoreContainer 22249946
[junit4:junit4]   2> 1548091 T4412 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369747946874\'
[junit4:junit4]   2> 1548092 T4412 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369747946874\'
[junit4:junit4]   2> 1548212 T4412 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369747946874
[junit4:junit4]   2> 1548215 T4412 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369747946874\collection1
[junit4:junit4]   2> 1548216 T4412 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369747946874\collection1\conf
[junit4:junit4]   2> 1548417 T4412 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369747946874\collection1\conf\xslt
[junit4:junit4]   2> 1548425 T4412 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369747946874\collection1\lib
[junit4:junit4]   2> 1548428 T4412 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369747946874\collection1\lib\classes
[junit4:junit4]   2> 1548434 T4412 oasc.SolrCoreDiscoverer.addCore Discovered properties file C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369747946874\conf\core.properties, adding to cores
[junit4:junit4]   2> 1548449 T4412 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 1548449 T4412 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 1548449 T4412 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 1548449 T4412 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 1548449 T4412 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 1548450 T4412 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 1548451 T4412 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 1548451 T4412 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 1548451 T4412 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 1548452 T4412 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 1548460 T4412 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 1548460 T4412 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50126/solr
[junit4:junit4]   2> 1548460 T4412 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 1548463 T4412 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1548479 T4478 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1767e8f name:ZooKeeperConnection Watcher:127.0.0.1:50126 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1548480 T4412 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1548486 T4412 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 1548492 T4412 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1548496 T4480 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ae2c9e name:ZooKeeperConnection Watcher:127.0.0.1:50126/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1548496 T4412 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1548505 T4412 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1549510 T4412 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50163_
[junit4:junit4]   2> 1549527 T4412 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50163_
[junit4:junit4]   2> 1549541 T4439 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 1549543 T4452 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 1549543 T4434 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 1549544 T4466 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 1549545 T4480 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 1549566 T4412 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
[junit4:junit4]   2> 1549567 T4412 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 1549568 T4412 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 1550010 T4412 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 1550016 T4412 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50172
[junit4:junit4]   2> 1550016 T4412 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 1550017 T4412 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1550017 T4412 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786
[junit4:junit4]   2> 1550017 T4412 oasc.CoreContainer$Initializer.initialize looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\solr.xml
[junit4:junit4]   2> 1550017 T4412 oasc.CoreContainer.<init> New CoreContainer 7571634
[junit4:junit4]   2> 1550019 T4412 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\'
[junit4:junit4]   2> 1550019 T4412 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\'
[junit4:junit4]   2> 1550124 T4412 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786
[junit4:junit4]   2> 1550126 T4412 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\collection1
[junit4:junit4]   2> 1550128 T4412 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\collection1\conf
[junit4:junit4]   2> 1550326 T4412 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\collection1\conf\xslt
[junit4:junit4]   2> 1550333 T4412 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\collection1\lib
[junit4:junit4]   2> 1550336 T4412 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\collection1\lib\classes
[junit4:junit4]   2> 1550342 T4412 oasc.SolrCoreDiscoverer.addCore Discovered properties file C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\conf\core.properties, adding to cores
[junit4:junit4]   2> 1550357 T4412 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 1550357 T4412 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 1550357 T4412 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 1550358 T4412 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 1550358 T4412 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 1550359 T4412 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 1550360 T4412 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 1550360 T4412 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 1550360 T4412 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 1550361 T4412 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 1550368 T4412 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 1550369 T4412 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50126/solr
[junit4:junit4]   2> 1550369 T4412 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 1550370 T4412 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1550387 T4492 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@143c335 name:ZooKeeperConnection Watcher:127.0.0.1:50126 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1550387 T4412 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1550393 T4412 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 1550400 T4412 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1550403 T4494 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b146 name:ZooKeeperConnection Watcher:127.0.0.1:50126/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1550403 T4412 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1550411 T4412 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1551423 T4412 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50172_
[junit4:junit4]   2> 1551434 T4412 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50172_
[junit4:junit4]   2> 1551440 T4480 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 1551441 T4434 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 1551442 T4452 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 1551442 T4494 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 1551443 T4466 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 1551444 T4439 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 1551455 T4412 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
[junit4:junit4]   2> 1551456 T4412 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 1551456 T4412 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 1551465 T4412 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 1551474 T4447 oasha.CollectionsHandler.handleCreateAction Creating Collection : numShards=2&name=nodes_used_collection&replicationFactor=2&action=CREATE&wt=javabin&version=2
[junit4:junit4]   2> 1551477 T4434 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4]   2> 1551481 T4436 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
[junit4:junit4]   2> 	  "operation":"createcollection",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "maxShardsPerNode":null,
[junit4:junit4]   2> 	  "createNodeSet":null,
[junit4:junit4]   2> 	  "name":"nodes_used_collection",
[junit4:junit4]   2> 	  "replicationFactor":"2"}
[junit4:junit4]   2> 1551481 T4436 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard1_replica1 as part of slice shard1 of collection nodes_used_collection on 127.0.0.1:50154_
[junit4:junit4]   2> 1551482 T4436 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard1_replica2 as part of slice shard1 of collection nodes_used_collection on 127.0.0.1:50133_
[junit4:junit4]   2> 1551482 T4436 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard2_replica1 as part of slice shard2 of collection nodes_used_collection on 127.0.0.1:50145_
[junit4:junit4]   2> 1551484 T4436 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard2_replica2 as part of slice shard2 of collection nodes_used_collection on 127.0.0.1:50172_
[junit4:junit4]   2> 1551485 T4460 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard1_replica1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369747944833\nodes_used_collection_shard1_replica1
[junit4:junit4]   2> 1551486 T4460 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4]   2> 1551486 T4428 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard1_replica2' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369747941658\nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 1551487 T4428 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4]   2> 1551487 T4443 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard2_replica1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369747942834\nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 1551488 T4443 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4]   2> 1551487 T4486 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard2_replica2' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\nodes_used_collection_shard2_replica2
[junit4:junit4]   2> 1551489 T4486 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4]   2> 1551489 T4460 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4]   2> 1551490 T4460 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1551490 T4443 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4]   2> 1551490 T4428 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4]   2> 1551492 T4428 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1551492 T4486 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4]   2> 1551493 T4486 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1551493 T4460 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 1551494 T4460 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4]   2> 1551492 T4443 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1551494 T4428 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 1551496 T4428 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4]   2> 1551496 T4486 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 1551497 T4486 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4]   2> 1551500 T4443 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 1551501 T4443 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4]   2> 1551502 T4460 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4]   2> 1551505 T4486 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4]   2> 1551506 T4428 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4]   2> 1551510 T4486 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\nodes_used_collection_shard2_replica2\'
[junit4:junit4]   2> 1551511 T4443 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4]   2> 1551510 T4460 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369747944833\nodes_used_collection_shard1_replica1\'
[junit4:junit4]   2> 1551517 T4443 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369747942834\nodes_used_collection_shard2_replica1\'
[junit4:junit4]   2> 1551518 T4428 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369747941658\nodes_used_collection_shard1_replica2\'
[junit4:junit4]   2> 1551603 T4460 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1551607 T4486 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1551679 T4443 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1551717 T4428 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1551738 T4486 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1551759 T4443 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1551788 T4460 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1551790 T4443 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1551795 T4443 oass.IndexSchema.readSchema [nodes_used_collection_shard2_replica1] Schema name=test
[junit4:junit4]   2> 1551800 T4486 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1551806 T4460 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1551807 T4486 oass.IndexSchema.readSchema [nodes_used_collection_shard2_replica2] Schema name=test
[junit4:junit4]   2> 1551811 T4460 oass.IndexSchema.readSchema [nodes_used_collection_shard1_replica1] Schema name=test
[junit4:junit4]   2> 1551857 T4428 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1551868 T4428 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1551874 T4428 oass.IndexSchema.readSchema [nodes_used_collection_shard1_replica2] Schema name=test
[junit4:junit4]   2> 1552965 T4443 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1552976 T4443 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1552980 T4443 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1553341 T4460 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1553349 T4460 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1553353 T4486 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1553354 T4460 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1553361 T4486 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1553366 T4486 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1553424 T4443 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1553431 T4443 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1553437 T4443 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1553441 T4460 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1553441 T4443 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1553442 T4443 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1553442 T4443 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1553447 T4460 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1553450 T4443 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1553450 T4443 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1553450 T4443 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1553451 T4443 oasc.SolrCore.<init> [nodes_used_collection_shard2_replica1] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369747942834\nodes_used_collection_shard2_replica1\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369747942834\nodes_used_collection_shard2_replica1\data\
[junit4:junit4]   2> 1553451 T4443 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6909db
[junit4:junit4]   2> 1553451 T4443 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1553451 T4443 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369747942834\nodes_used_collection_shard2_replica1\data\
[junit4:junit4]   2> 1553453 T4443 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369747942834\nodes_used_collection_shard2_replica1\data\index/
[junit4:junit4]   2> 1553453 T4443 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard2_replica1] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369747942834\nodes_used_collection_shard2_replica1\data\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1553453 T4428 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1553454 T4443 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369747942834\nodes_used_collection_shard2_replica1\data\index
[junit4:junit4]   2> 1553458 T4460 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1553461 T4443 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1f33676 lockFactory=org.apache.lucene.store.NativeFSLockFactory@bbd508),segFN=segments_1,generation=1}
[junit4:junit4]   2> 1553461 T4443 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 1553466 T4443 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1553466 T4443 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1553468 T4443 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1553468 T4443 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1553469 T4443 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1553469 T4443 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1553469 T4443 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1553470 T4443 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1553470 T4443 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1553475 T4428 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1553480 T4428 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1553482 T4443 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1553485 T4486 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1553485 T4460 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1553486 T4460 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1553486 T4460 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1553488 T4460 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1553489 T4460 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1553489 T4460 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1553489 T4460 oasc.SolrCore.<init> [nodes_used_collection_shard1_replica1] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369747944833\nodes_used_collection_shard1_replica1\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369747944833\nodes_used_collection_shard1_replica1\data\
[junit4:junit4]   2> 1553490 T4460 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6909db
[junit4:junit4]   2> 1553490 T4460 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1553492 T4460 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369747944833\nodes_used_collection_shard1_replica1\data\
[junit4:junit4]   2> 1553492 T4460 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369747944833\nodes_used_collection_shard1_replica1\data\index/
[junit4:junit4]   2> 1553492 T4460 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard1_replica1] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369747944833\nodes_used_collection_shard1_replica1\data\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1553493 T4460 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369747944833\nodes_used_collection_shard1_replica1\data\index
[junit4:junit4]   2> 1553494 T4443 oass.SolrIndexSearcher.<init> Opening Searcher@1bdf688 main
[junit4:junit4]   2> 1553495 T4443 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369747942834\nodes_used_collection_shard2_replica1\data\tlog
[junit4:junit4]   2> 1553497 T4443 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1553497 T4443 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1553502 T4486 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1553506 T4460 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@13ffd11 lockFactory=org.apache.lucene.store.NativeFSLockFactory@a8569a),segFN=segments_1,generation=1}
[junit4:junit4]   2> 1553506 T4460 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 1553510 T4460 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1553510 T4460 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1553511 T4460 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1553511 T4460 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1553512 T4460 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1553513 T4460 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1553513 T4460 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1553514 T4460 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1553514 T4460 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1553515 T4486 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1553520 T4486 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1553520 T4486 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1553520 T4486 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1553522 T4500 oasc.SolrCore.registerSearcher [nodes_used_collection_shard2_replica1] Registered new searcher Searcher@1bdf688 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1553524 T4486 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1553524 T4486 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1553525 T4486 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1553524 T4443 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica1 state=down
[junit4:junit4]   2> 1553525 T4486 oasc.SolrCore.<init> [nodes_used_collection_shard2_replica2] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\nodes_used_collection_shard2_replica2\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\nodes_used_collection_shard2_replica2\data\
[junit4:junit4]   2> 1553526 T4486 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6909db
[junit4:junit4]   2> 1553527 T4486 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1553528 T4486 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\nodes_used_collection_shard2_replica2\data\
[junit4:junit4]   2> 1553528 T4486 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\nodes_used_collection_shard2_replica2\data\index/
[junit4:junit4]   2> 1553528 T4486 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard2_replica2] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\nodes_used_collection_shard2_replica2\data\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1553528 T4486 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\nodes_used_collection_shard2_replica2\data\index
[junit4:junit4]   2> 1553535 T4460 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1553536 T4486 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@838e14 lockFactory=org.apache.lucene.store.NativeFSLockFactory@19df105),segFN=segments_1,generation=1}
[junit4:junit4]   2> 1553536 T4486 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 1553540 T4486 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1553541 T4486 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1553541 T4486 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1553542 T4486 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1553543 T4486 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1553543 T4486 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1553543 T4460 oass.SolrIndexSearcher.<init> Opening Searcher@1f8d2d7 main
[junit4:junit4]   2> 1553544 T4460 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369747944833\nodes_used_collection_shard1_replica1\data\tlog
[junit4:junit4]   2> 1553544 T4486 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1553546 T4486 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1553547 T4486 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1553547 T4460 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1553547 T4460 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1553552 T4460 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica1 state=down
[junit4:junit4]   2> 1553553 T4428 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1553554 T4501 oasc.SolrCore.registerSearcher [nodes_used_collection_shard1_replica1] Registered new searcher Searcher@1f8d2d7 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1553560 T4486 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1553564 T4486 oass.SolrIndexSearcher.<init> Opening Searcher@1db1253 main
[junit4:junit4]   2> 1553564 T4486 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\nodes_used_collection_shard2_replica2\data\tlog
[junit4:junit4]   2> 1553566 T4486 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1553567 T4486 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1553571 T4486 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica2 state=down
[junit4:junit4]   2> 1553571 T4428 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1553572 T4502 oasc.SolrCore.registerSearcher [nodes_used_collection_shard2_replica2] Registered new searcher Searcher@1db1253 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1553579 T4428 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1553582 T4428 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1553582 T4428 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1553583 T4428 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1553584 T4428 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1553585 T4428 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1553585 T4428 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1553586 T4428 oasc.SolrCore.<init> [nodes_used_collection_shard1_replica2] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369747941658\nodes_used_collection_shard1_replica2\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369747941658\nodes_used_collection_shard1_replica2\data\
[junit4:junit4]   2> 1553586 T4428 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6909db
[junit4:junit4]   2> 1553587 T4428 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1553587 T4428 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369747941658\nodes_used_collection_shard1_replica2\data\
[junit4:junit4]   2> 1553588 T4428 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369747941658\nodes_used_collection_shard1_replica2\data\index/
[junit4:junit4]   2> 1553588 T4428 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard1_replica2] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369747941658\nodes_used_collection_shard1_replica2\data\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1553589 T4428 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369747941658\nodes_used_collection_shard1_replica2\data\index
[junit4:junit4]   2> 1553595 T4428 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@f527d6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@10d8995),segFN=segments_1,generation=1}
[junit4:junit4]   2> 1553595 T4428 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 1553600 T4428 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1553600 T4428 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1553601 T4428 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1553602 T4428 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1553602 T4428 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1553603 T4428 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1553603 T4428 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1553604 T4428 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1553605 T4428 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1553616 T4428 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1553623 T4428 oass.SolrIndexSearcher.<init> Opening Searcher@e90fde main
[junit4:junit4]   2> 1553623 T4428 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369747941658\nodes_used_collection_shard1_replica2\data\tlog
[junit4:junit4]   2> 1553625 T4428 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1553625 T4428 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1553631 T4503 oasc.SolrCore.registerSearcher [nodes_used_collection_shard1_replica2] Registered new searcher Searcher@e90fde main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1553632 T4428 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica2 state=down
[junit4:junit4]   2> 1554673 T4435 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 1554675 T4435 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard2_replica1",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:50145_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:50145"}
[junit4:junit4]   2> 1554675 T4435 oasc.Overseer$ClusterStateUpdater.createCollection Create collection nodes_used_collection with numShards 2
[junit4:junit4]   2> 1554691 T4435 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard1_replica1",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:50154_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:50154"}
[junit4:junit4]   2> 1554705 T4435 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard2_replica2",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:50172_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:50172"}
[junit4:junit4]   2> 1554723 T4435 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard1_replica2",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:50133_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:50133"}
[junit4:junit4]   2> 1554745 T4439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1554747 T4452 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1554746 T4480 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1554747 T4434 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1554747 T4466 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1554747 T4494 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1555536 T4443 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 1555536 T4443 oasc.ZkController.register Register replica - core:nodes_used_collection_shard2_replica1 address:http://127.0.0.1:50145 collection:nodes_used_collection shard:shard2
[junit4:junit4]   2> 1555539 T4443 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leader_elect/shard2/election
[junit4:junit4]   2> 1555558 T4460 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard1_replica1
[junit4:junit4]   2> 1555558 T4460 oasc.ZkController.register Register replica - core:nodes_used_collection_shard1_replica1 address:http://127.0.0.1:50154 collection:nodes_used_collection shard:shard1
[junit4:junit4]   2> 1555561 T4460 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leader_elect/shard1/election
[junit4:junit4]   2> 1555575 T4486 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard2_replica2
[junit4:junit4]   2> 1555576 T4486 oasc.ZkController.register Register replica - core:nodes_used_collection_shard2_replica2 address:http://127.0.0.1:50172 collection:nodes_used_collection shard:shard2
[junit4:junit4]   2> 1555592 T4443 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 1555607 T4460 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 1555613 T4443 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 1555615 T4443 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 1555617 T4443 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:50145/nodes_used_collection_shard2_replica1/
[junit4:junit4]   2> 1555618 T4443 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard2_replica1 url=http://127.0.0.1:50145 START replicas=[http://127.0.0.1:50172/nodes_used_collection_shard2_replica2/] nUpdates=100
[junit4:junit4]   2> 1555620 T4443 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard2_replica1 url=http://127.0.0.1:50145 DONE.  We have no versions.  sync failed.
[junit4:junit4]   2> 1555621 T4443 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next canidate
[junit4:junit4]   2> 1555622 T4443 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4:junit4]   2> 1555624 T4443 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:50145/nodes_used_collection_shard2_replica1/
[junit4:junit4]   2> 1555625 T4443 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leaders/shard2
[junit4:junit4]   2> 1555623 T4460 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up: total=2 found=1 timeoutin=179996
[junit4:junit4]   2> 1555635 T4428 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 1555635 T4428 oasc.ZkController.register Register replica - core:nodes_used_collection_shard1_replica2 address:http://127.0.0.1:50133 collection:nodes_used_collection shard:shard1
[junit4:junit4]   2> ASYNC  NEW_CORE C5547 name=nodes_used_collection_shard2_replica2 org.apache.solr.core.SolrCore@1b30d8e url=http://127.0.0.1:50172/nodes_used_collection_shard2_replica2 node=127.0.0.1:50172_ C5547_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica2 props:{shard=shard2, state=down, core=nodes_used_collection_shard2_replica2, collection=nodes_used_collection, node_name=127.0.0.1:50172_, base_url=http://127.0.0.1:50172}
[junit4:junit4]   2> 1555648 T4489 C5547 P50172 oasc.SolrCore.execute [nodes_used_collection_shard2_replica2] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1 
[junit4:junit4]   2> 1556128 T4460 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 1556130 T4460 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 1556130 T4460 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:50154/nodes_used_collection_shard1_replica1/
[junit4:junit4]   2> 1556132 T4460 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard1_replica1 url=http://127.0.0.1:50154 START replicas=[http://127.0.0.1:50133/nodes_used_collection_shard1_replica2/] nUpdates=100
[junit4:junit4]   2> 1556133 T4460 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard1_replica1 url=http://127.0.0.1:50154 DONE.  We have no versions.  sync failed.
[junit4:junit4]   2> 1556134 T4460 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next canidate
[junit4:junit4]   2> 1556135 T4460 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4:junit4]   2> 1556136 T4460 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:50154/nodes_used_collection_shard1_replica1/
[junit4:junit4]   2> 1556137 T4460 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leaders/shard1
[junit4:junit4]   2> ASYNC  NEW_CORE C5548 name=nodes_used_collection_shard1_replica2 org.apache.solr.core.SolrCore@12f7054 url=http://127.0.0.1:50133/nodes_used_collection_shard1_replica2 node=127.0.0.1:50133_ C5548_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica2 props:{shard=shard1, state=down, core=nodes_used_collection_shard1_replica2, collection=nodes_used_collection, node_name=127.0.0.1:50133_, base_url=http://127.0.0.1:50133}
[junit4:junit4]   2> 1556158 T4425 C5548 P50133 oasc.SolrCore.execute [nodes_used_collection_shard1_replica2] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1 
[junit4:junit4]   2> 1556295 T4435 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 1556371 T4480 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1556372 T4434 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1556372 T4439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1556373 T4466 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1556375 T4452 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1556373 T4494 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1556418 T4443 oasc.ZkController.register We are http://127.0.0.1:50145/nodes_used_collection_shard2_replica1/ and leader is http://127.0.0.1:50145/nodes_used_collection_shard2_replica1/
[junit4:junit4]   2> 1556418 T4443 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard2_replica1 baseURL=http://127.0.0.1:50145
[junit4:junit4]   2> 1556419 T4443 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 1556419 T4443 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica1 state=active
[junit4:junit4]   2> 1556429 T4443 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 1556430 T4443 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard2&name=nodes_used_collection_shard2_replica1&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=4943 
[junit4:junit4]   2> 1556465 T4460 oasc.ZkController.register We are http://127.0.0.1:50154/nodes_used_collection_shard1_replica1/ and leader is http://127.0.0.1:50154/nodes_used_collection_shard1_replica1/
[junit4:junit4]   2> 1556465 T4460 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard1_replica1 baseURL=http://127.0.0.1:50154
[junit4:junit4]   2> 1556466 T4460 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 1556466 T4460 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica1 state=active
[junit4:junit4]   2> 1556473 T4460 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 1556475 T4460 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard1&name=nodes_used_collection_shard1_replica1&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=4990 
[junit4:junit4]   2> 1556603 T4486 oasc.ZkController.register We are http://127.0.0.1:50172/nodes_used_collection_shard2_replica2/ and leader is http://127.0.0.1:50145/nodes_used_collection_shard2_replica1/
[junit4:junit4]   2> 1556605 T4486 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard2_replica2 baseURL=http://127.0.0.1:50172
[junit4:junit4]   2> 1556605 T4486 oasc.ZkController.checkRecovery Core needs to recover:nodes_used_collection_shard2_replica2
[junit4:junit4]   2> 1556606 T4486 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> 1556608 T4506 C5547 P50172 oasc.RecoveryStrategy.run Starting recovery process.  core=nodes_used_collection_shard2_replica2 recoveringAfterStartup=true
[junit4:junit4]   2> 1556609 T4506 C5547 P50172 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 1556610 T4506 C5547 P50172 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica2 state=recovering
[junit4:junit4]   2> 1556610 T4486 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 1556616 T4506 C5547 P50172 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 1556617 T4486 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard2&name=nodes_used_collection_shard2_replica2&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=5130 
[junit4:junit4]   2> 1556648 T4444 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:50172__nodes_used_collection_shard2_replica2, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4]   2> 1556651 T4428 oasc.ZkController.register We are http://127.0.0.1:50133/nodes_used_collection_shard1_replica2/ and leader is http://127.0.0.1:50154/nodes_used_collection_shard1_replica1/
[junit4:junit4]   2> 1556652 T4428 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard1_replica2 baseURL=http://127.0.0.1:50133
[junit4:junit4]   2> 1556652 T4428 oasc.ZkController.checkRecovery Core needs to recover:nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 1556652 T4428 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C5549 name=nodes_used_collection_shard1_replica2 org.apache.solr.core.SolrCore@12f7054 url=http://127.0.0.1:50133/nodes_used_collection_shard1_replica2 node=127.0.0.1:50133_ C5549_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica2 props:{shard=shard1, state=down, core=nodes_used_collection_shard1_replica2, collection=nodes_used_collection, node_name=127.0.0.1:50133_, base_url=http://127.0.0.1:50133}
[junit4:junit4]   2> 1556653 T4507 C5549 P50133 oasc.RecoveryStrategy.run Starting recovery process.  core=nodes_used_collection_shard1_replica2 recoveringAfterStartup=true
[junit4:junit4]   2> 1556653 T4507 C5549 P50133 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 1556653 T4428 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 1556654 T4507 C5549 P50133 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica2 state=recovering
[junit4:junit4]   2> 1556655 T4428 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard1&name=nodes_used_collection_shard1_replica2&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=5169 
[junit4:junit4]   2> 1556656 T4436 oasc.OverseerCollectionProcessor.createCollection Finished create command on all shards for collection: nodes_used_collection
[junit4:junit4]   2> 1556658 T4507 C5549 P50133 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 1556669 T4457 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:50133__nodes_used_collection_shard1_replica2, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4]   2> 1556670 T4452 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work/qnr-0000000000 state: SyncConnected type NodeDataChanged
[junit4:junit4]   2> 1556673 T4434 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4]   2> 1556674 T4436 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Message id:/overseer/collection-queue-work/qn-0000000000 complete, response:{success={null={responseHeader={status=0,QTime=4943},core=nodes_used_collection_shard2_replica1},null={responseHeader={status=0,QTime=4990},core=nodes_used_collection_shard1_replica1},null={responseHeader={status=0,QTime=5130},core=nodes_used_collection_shard2_replica2},null={responseHeader={status=0,QTime=5169},core=nodes_used_collection_shard1_replica2}}}
[junit4:junit4]   2> 1556677 T4447 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={numShards=2&name=nodes_used_collection&replicationFactor=2&action=CREATE&wt=javabin&version=2} status=0 QTime=5203 
[junit4:junit4]   2> 1556678 T4412 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 1556680 T4412 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1556685 T4509 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@dc7914 name:ZooKeeperConnection Watcher:127.0.0.1:50126/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1556686 T4412 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1556688 T4412 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1556694 T4412 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 1556697 T4412 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1556701 T4511 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@83d435 name:ZooKeeperConnection Watcher:127.0.0.1:50126/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1556701 T4412 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1556704 T4412 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1556715 T4446 oasha.CollectionsHandler.handleCreateAction Creating Collection : numShards=4&maxShardsPerNode=4&name=awholynewcollection_0&replicationFactor=4&action=CREATE&wt=javabin&version=2
[junit4:junit4]   2> 1556718 T4434 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4]   2> 1556721 T4434 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4]   2> 1556722 T4436 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000002 message:{
[junit4:junit4]   2> 	  "operation":"createcollection",
[junit4:junit4]   2> 	  "numShards":"4",
[junit4:junit4]   2> 	  "maxShardsPerNode":"4",
[junit4:junit4]   2> 	  "createNodeSet":null,
[junit4:junit4]   2> 	  "name":"awholynewcollection_0",
[junit4:junit4]   2> 	  "replicationFactor":"4"}
[junit4:junit4]   2> 1556722 T4436 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica1 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:50133_
[junit4:junit4]   2> 1556722 T4436 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica2 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:50172_
[junit4:junit4]   2> 1556723 T4436 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica3 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:50154_
[junit4:junit4]   2> 1556724 T4436 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica4 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:50163_
[junit4:junit4]   2> 1556724 T4436 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard2_replica1 as part of slice shard2 of collection awholynewcollection_0 on 127.0.0.1:50145_
[junit4:junit4]   2> 1556726 T4436 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard2_replica2 as part of slice shard2 of collection awholynewcollection_0 on 127.0.0.1:50133_
[junit4:junit4]   2> 1556728 T4472 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica4' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369747946874\awholynewcollection_0_shard1_replica4
[junit4:junit4]   2> 1556728 T4472 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 1556727 T4490 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica2' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\awholynewcollection_0_shard1_replica2
[junit4:junit4]   2> 1556733 T4490 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 1556727 T4427 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369747941658\awholynewcollection_0_shard1_replica1
[junit4:junit4]   2> 1556734 T4427 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 1556726 T4458 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica3' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369747944833\awholynewcollection_0_shard1_replica3
[junit4:junit4]   2> 1556735 T4458 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 1556734 T4490 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4]   2> 1556736 T4490 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1556733 T4472 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4]   2> 1556736 T4458 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4]   2> 1556736 T4458 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1556733 T4429 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard2_replica2' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369747941658\awholynewcollection_0_shard2_replica2
[junit4:junit4]   2> 1556740 T4429 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 1556732 T4448 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard2_replica1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369747942834\awholynewcollection_0_shard2_replica1
[junit4:junit4]   2> 1556743 T4448 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 1556744 T4429 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4]   2> 1556745 T4429 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1556728 T4436 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard2_replica3 as part of slice shard2 of collection awholynewcollection_0 on 127.0.0.1:50172_
[junit4:junit4]   2> 1556746 T4448 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4]   2> 1556743 T4458 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 1556748 T4458 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4]   2> 1556740 T4490 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 1556751 T4490 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4]   2> 1556736 T4472 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1556735 T4427 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4]   2> 1556754 T4427 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1556753 T4487 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard2_replica3' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\awholynewcollection_0_shard2_replica3
[junit4:junit4]   2> 1556755 T4487 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 1556748 T4448 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1556748 T4436 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard2_replica4 as part of slice shard2 of collection awholynewcollection_0 on 127.0.0.1:50154_
[junit4:junit4]   2> 1556754 T4429 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 1556757 T4429 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4]   2> 1556760 T4472 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 1556760 T4472 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4]   2> 1556757 T4436 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard3_replica1 as part of slice shard3 of collection awholynewcollection_0 on 127.0.0.1:50163_
[junit4:junit4]   2> 1556760 T4427 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 1556762 T4427 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4]   2> 1556760 T4487 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 1556764 T4487 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 1556765 T4490 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 1556759 T4459 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard2_replica4' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369747944833\awholynewcollection_0_shard2_replica4
[junit4:junit4]   2> 1556768 T4459 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 1556767 T4473 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard3_replica1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369747946874\awholynewcollection_0_shard3_replica1
[junit4:junit4]   2> 1556769 T4473 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 1556763 T4448 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4]   2> 1556771 T4472 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 1556763 T4458 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 1556773 T4487 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\awholynewcollection_0_shard2_replica3\'
[junit4:junit4]   2> 1556762 T4436 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard3_replica2 as part of slice shard3 of collection awholynewcollection_0 on 127.0.0.1:50145_
[junit4:junit4]   2> 1556774 T4429 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 1556776 T4490 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\awholynewcollection_0_shard1_replica2\'
[junit4:junit4]   2> 1556778 T4436 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard3_replica3 as part of slice shard3 of collection awholynewcollection_0 on 127.0.0.1:50133_
[junit4:junit4]   2> 1556778 T4436 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard3_replica4 as part of slice shard3 of collection awholynewcollection_0 on 127.0.0.1:50172_
[junit4:junit4]   2> 1556779 T4436 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard4_replica1 as part of slice shard4 of collection awholynewcollection_0 on 127.0.0.1:50154_
[junit4:junit4]   2> 1556780 T4436 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard4_replica2 as part of slice shard4 of collection awholynewcollection_0 on 127.0.0.1:50163_
[junit4:junit4]   2> 1556781 T4436 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard4_replica3 as part of slice shard4 of collection awholynewcollection_0 on 127.0.0.1:50145_
[junit4:junit4]   2> 1556781 T4436 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard4_replica4 as part of slice shard4 of collection awholynewcollection_0 on 127.0.0.1:50133_
[junit4:junit4]   2> 1556776 T4473 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 1556784 T4473 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 1556784 T4472 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369747946874\awholynewcollection_0_shard1_replica4\'
[junit4:junit4]   2> 1556776 T4459 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 1556785 T4459 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 1556783 T4427 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 1556789 T4458 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369747944833\awholynewcollection_0_shard1_replica3\'
[junit4:junit4]   2> 1556794 T4489 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard3_replica4' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\awholynewcollection_0_shard3_replica4
[junit4:junit4]   2> 1556794 T4489 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 1556790 T4430 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard3_replica3' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369747941658\awholynewcollection_0_shard3_replica3
[junit4:junit4]   2> 1556796 T4430 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 1556797 T4474 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard4_replica2' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369747946874\awholynewcollection_0_shard4_replica2
[junit4:junit4]   2> 1556797 T4474 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 1556793 T4448 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 1556798 T4429 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369747941658\awholynewcollection_0_shard2_replica2\'
[junit4:junit4]   2> 1556792 T4425 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard4_replica4' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369747941658\awholynewcollection_0_shard4_replica4
[junit4:junit4]   2> 1556805 T4425 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 1556806 T4447 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard4_replica3' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369747942834\awholynewcollection_0_shard4_replica3
[junit4:junit4]   2> 1556806 T4447 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 1556791 T4462 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard4_replica1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369747944833\awholynewcollection_0_shard4_replica1
[junit4:junit4]   2> 1556821 T4462 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 1556827 T4489 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 1556827 T4489 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 1556830 T4473 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369747946874\awholynewcollection_0_shard3_replica1\'
[junit4:junit4]   2> 1556832 T4459 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369747944833\awholynewcollection_0_shard2_replica4\'
[junit4:junit4]   2> 1556844 T4462 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 1556845 T4462 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 1556804 T4443 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard3_replica2' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369747942834\awholynewcollection_0_shard3_replica2
[junit4:junit4]   2> 1556852 T4443 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 1556827 T4474 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 1556853 T4474 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 1556827 T4430 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 1556854 T4430 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 1556858 T4447 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 1556858 T4447 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 1556864 T4427 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369747941658\awholynewcollection_0_shard1_replica1\'
[junit4:junit4]   2> 1556873 T4448 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369747942834\awholynewcollection_0_shard2_replica1\'
[junit4:junit4]   2> 1556884 T4425 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 1556885 T4425 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 1556886 T4443 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 1556886 T4443 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 1556911 T4462 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369747944833\awholynewcollection_0_shard4_replica1\'
[junit4:junit4]   2> 1556914 T4489 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\awholynewcollection_0_shard3_replica4\'
[junit4:junit4]   2> 1556915 T4474 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369747946874\awholynewcollection_0_shard4_replica2\'
[junit4:junit4]   2> 1556915 T4430 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369747941658\awholynewcollection_0_shard3_replica3\'
[junit4:junit4]   2> 1556915 T4447 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369747942834\awholynewcollection_0_shard4_replica3\'
[junit4:junit4]   2> 1556920 T4443 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369747942834\awholynewcollection_0_shard3_replica2\'
[junit4:junit4]   2> 1556986 T4490 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1557005 T4429 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1557100 T4487 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1557110 T4473 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1557168 T4425 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369747941658\awholynewcollection_0_shard4_replica4\'
[junit4:junit4]   2> 1557191 T4459 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1557206 T4427 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1557291 T4427 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1557293 T4448 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1557331 T4427 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1557331 T4472 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1557365 T4427 oass.IndexSchema.readSchema [awholynewcollection_0_shard1_replica1] Schema name=test
[junit4:junit4]   2> 1557382 T4459 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1557395 T4429 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1557407 T4487 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1557409 T4458 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1557419 T4459 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1557419 T4429 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1557420 T4487 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1557435 T4429 oass.IndexSchema.readSchema [awholynewcollection_0_shard2_replica2] Schema name=test
[junit4:junit4]   2> 1557462 T4443 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1557533 T4448 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1557535 T4490 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1557645 T4448 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1557646 T4490 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1557653 T4487 oass.IndexSchema.readSchema [awholynewcollection_0_shard2_replica3] Schema name=test
[junit4:junit4]   2> 1557662 T4448 oass.IndexSchema.readSchema [awholynewcollection_0_shard2_replica1] Schema name=test
[junit4:junit4]   2> 1557745 T4474 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1557765 T4473 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1557838 T4472 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1557899 T4462 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1557912 T4472 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1557912 T4473 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1557918 T4472 oass.IndexSchema.readSchema [awholynewcollection_0_shard1_replica4] Schema name=test
[junit4:junit4]   2> 1557919 T4473 oass.IndexSchema.readSchema [awholynewcollection_0_shard3_replica1] Schema name=test
[junit4:junit4]   2> 1558017 T4490 oass.IndexSchema.readSchema [awholynewcollection_0_shard1_replica2] Schema name=test
[junit4:junit4]   2> 1558061 T4459 oass.IndexSchema.readSchema [awholynewcollection_0_shard2_replica4] Schema name=test
[junit4:junit4]   2> 1558071 T4435 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 1558073 T4435 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:50145__nodes_used_collection_shard2_replica1",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard2_replica1",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:50145_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:50145"}
[junit4:junit4]   2> 1558091 T4435 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:50154__nodes_used_collection_shard1_replica1",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard1_replica1",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:50154_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:50154"}
[junit4:junit4]   2> 1558096 T4430 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1558097 T4447 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1558116 T4435 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:50172__nodes_used_collection_shard2_replica2",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard2_replica2",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:50172_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:50172"}
[junit4:junit4]   2> 1558167 T4474 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1558157 T4458 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1558409 T4430 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1558370 T4447 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1558616 T4489 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1558773 T4447 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1558774 T4474 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1558777 T4458 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1558781 T4458 oass.IndexSchema.readSchema [awholynewcollection_0_shard1_replica3] Schema name=test
[junit4:junit4]   2> 1558805 T4430 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1558816 T4474 oass.IndexSchema.readSchema [awholynewcollection_0_shard4_replica2] Schema name=test
[junit4:junit4]   2> 1558894 T4435 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:50133__nodes_used_collection_shard1_replica2",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard1_replica2",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:50133_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:50133"}
[junit4:junit4]   2> 1558898 T4447 oass.IndexSchema.readSchema [awholynewcollection_0_shard4_replica3] Schema name=test
[junit4:junit4]   2> 1558907 T4430 oass.IndexSchema.readSchema [awholynewcollection_0_shard3_replica3] Schema name=test
[junit4:junit4]   2> 1558915 T4480 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1558915 T4434 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1558916 T4494 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1558916 T4466 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1558916 T4452 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1558917 T4439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1558918 T4509 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1558927 T4511 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1558928 T4489 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1558943 T4489 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1558962 T4489 oass.IndexSchema.readSchema [awholynewcollection_0_shard3_replica4] Schema name=test
[junit4:junit4]   2> 1559303 T4443 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1559308 T4443 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1559366 T4462 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1559408 T4443 oass.IndexSchema.readSchema [awholynewcollection_0_shard3_replica2] Schema name=test
[junit4:junit4]   2> 1559433 T4462 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1559455 T4462 oass.IndexSchema.readSchema [awholynewcollection_0_shard4_replica1] Schema name=test
[junit4:junit4]   2> 1559475 T4425 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1559777 T4444 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:50172__nodes_used_collection_shard2_replica2, state: recovering, checkLive: true, onlyIfLeader: true for: 3 seconds.
[junit4:junit4]   2> 1559777 T4444 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:50172__nodes_used_collection_shard2_replica2&state=recovering&nodeName=127.0.0.1:50172_&action=PREPRECOVERY&checkLive=true&core=nodes_used_collection_shard2_replica1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=3129 
[junit4:junit4]   2> 1559778 T4457 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:50133__nodes_used_collection_shard1_replica2, state: recovering, checkLive: true, onlyIfLeader: true for: 3 seconds.
[junit4:junit4]   2> 1559785 T4457 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:50133__nodes_used_collection_shard1_replica2&state=recovering&nodeName=127.0.0.1:50133_&action=PREPRECOVERY&checkLive=true&core=nodes_used_collection_shard1_replica1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=3116 
[junit4:junit4]   2> 1559986 T4425 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1560219 T4425 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1560325 T4425 oass.IndexSchema.readSchema [awholynewcollection_0_shard4_replica4] Schema name=test
[junit4:junit4]   2> ASYNC  NEW_CORE C5550 name=nodes_used_collection_shard1_replica2 org.apache.solr.core.SolrCore@12f7054 url=http://127.0.0.1:50133/nodes_used_collection_shard1_replica2 node=127.0.0.1:50133_ C5550_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica2 props:{shard=shard1, state=recovering, core=nodes_used_collection_shard1_replica2, collection=nodes_used_collection, node_name=127.0.0.1:50133_, base_url=http://127.0.0.1:50133}
[junit4:junit4]   2> 1561864 T4507 C5550 P50133 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:50154/nodes_used_collection_shard1_replica1/ core=nodes_used_collection_shard1_replica2 - recoveringAfterStartup=true
[junit4:junit4]   2> 1561865 T4507 C5550 P50133 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard1_replica2 url=http://127.0.0.1:50133 START replicas=[http://127.0.0.1:50154/nodes_used_collection_shard1_replica1/] nUpdates=100
[junit4:junit4]   2> ASYNC  NEW_CORE C5551 name=nodes_used_collection_shard2_replica2 org.apache.solr.core.SolrCore@1b30d8e url=http://127.0.0.1:50172/nodes_used_collection_shard2_replica2 node=127.0.0.1:50172_ C5551_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica2 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica2, collection=nodes_used_collection, node_name=127.0.0.1:50172_, base_url=http://127.0.0.1:50172}
[junit4:junit4]   2> 1561865 T4506 C5551 P50172 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:50145/nodes_used_collection_shard2_replica1/ core=nodes_used_collection_shard2_replica2 - recoveringAfterStartup=true
[junit4:junit4]   2> 1561865 T4506 C5551 P50172 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard2_replica2 url=http://127.0.0.1:50172 START replicas=[http://127.0.0.1:50145/nodes_used_collection_shard2_replica1/] nUpdates=100
[junit4:junit4]   2> 1561865 T4507 C5550 P50133 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 1561866 T4507 C5550 P50133 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 1561866 T4507 C5550 P50133 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 1561866 T4507 C5550 P50133 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 1561866 T4507 C5550 P50133 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 1561866 T4507 C5550 P50133 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:50154/nodes_used_collection_shard1_replica1/. core=nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 1561867 T4507 C5550 P50133 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 1561866 T4506 C5551 P50172 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 1561883 T4506 C5551 P50172 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=nodes_used_collection_shard2_replica2
[junit4:junit4]   2> 1561883 T4506 C5551 P50172 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=nodes_used_collection_shard2_replica2
[junit4:junit4]   2> 1561883 T4506 C5551 P50172 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=nodes_used_collection_shard2_replica2
[junit4:junit4]   2> 1561883 T4506 C5551 P50172 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 1561883 T4506 C5551 P50172 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:50145/nodes_used_collection_shard2_replica1/. core=nodes_used_collection_shard2_replica2
[junit4:junit4]   2> 1561883 T4506 C5551 P50172 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 1561947 T4472 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> ASYNC  NEW_CORE C5552 name=nodes_used_collection_shard1_replica1 org.apache.solr.core.SolrCore@10402f2 url=http://127.0.0.1:50154/nodes_used_collection_shard1_replica1 node=127.0.0.1:50154_ C5552_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica1 props:{shard=shard1, state=active, core=nodes_used_collection_shard1_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50154_, base_url=http://127.0.0.1:50154, leader=true}
[junit4:junit4]   2> 1562260 T4461 C5552 P50154 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> ASYNC  NEW_CORE C5553 name=nodes_used_collection_shard1_replica1 org.apache.solr.core.SolrCore@10402f2 url=http://127.0.0.1:50154/nodes_used_collection_shard1_replica1 node=127.0.0.1:50154_ C5553_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica1 props:{shard=shard1, state=active, core=nodes_used_collection_shard1_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50154_, base_url=http://127.0.0.1:50154, leader=true}
[junit4:junit4]   2> 1562392 T4457 C5553 P50154 oasc.SolrCore.execute [nodes_used_collection_shard1_replica1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=131 
[junit4:junit4]   2> 1562394 T4461 C5553 P50154 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@13ffd11 lockFactory=org.apache.lucene.store.NativeFSLockFactory@a8569a),segFN=segments_1,generation=1}
[junit4:junit4]   2> 1562427 T4461 C5553 P50154 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 1562456 T4461 C5553 P50154 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@13ffd11 lockFactory=org.apache.lucene.store.NativeFSLockFactory@a8569a),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@13ffd11 lockFactory=org.apache.lucene.store.NativeFSLockFactory@a8569a),segFN=segments_2,generation=2}
[junit4:junit4]   2> 1562456 T4461 C5553 P50154 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 1562463 T4461 C5553 P50154 oass.SolrIndexSearcher.<init> Opening Searcher@695530 realtime
[junit4:junit4]   2> 1562465 T4461 C5553 P50154 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 1562465 T4461 C5553 P50154 oasup.LogUpdateProcessor.finish [nodes_used_collection_shard1_replica1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 206
[junit4:junit4]   2> ASYNC  NEW_CORE C5554 name=nodes_used_collection_shard1_replica2 org.apache.solr.core.SolrCore@12f7054 url=http://127.0.0.1:50133/nodes_used_collection_shard1_replica2 node=127.0.0.1:50133_ C5554_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica2 props:{shard=shard1, state=recovering, core=nodes_used_collection_shard1_replica2, collection=nodes_used_collection, node_name=127.0.0.1:50133_, base_url=http://127.0.0.1:50133}
[junit4:junit4]   2> 1562466 T4507 C5554 P50133 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 1562465 T4472 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1562466 T4507 C5554 P50133 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> ASYNC  NEW_CORE C5555 name=nodes_used_collection_shard1_replica1 org.apache.solr.core.SolrCore@10402f2 url=http://127.0.0.1:50154/nodes_used_collection_shard1_replica1 node=127.0.0.1:50154_ C5555_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica1 props:{shard=shard1, state=active, core=nodes_used_collection_shard1_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50154_, base_url=http://127.0.0.1:50154, leader=true}
[junit4:junit4]   2> 1562516 T4457 C5555 P50154 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 1562517 T4457 C5555 P50154 oasc.SolrCore.execute [nodes_used_collection_shard1_replica1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=48 
[junit4:junit4]   2> ASYNC  NEW_CORE C5556 name=nodes_used_collection_shard1_replica2 org.apache.solr.core.SolrCore@12f7054 url=http://127.0.0.1:50133/nodes_used_collection_shard1_replica2 node=127.0.0.1:50133_ C5556_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica2 props:{shard=shard1, state=recovering, core=nodes_used_collection_shard1_replica2, collection=nodes_used_collection, node_name=127.0.0.1:50133_, base_url=http://127.0.0.1:50133}
[junit4:junit4]   2> 1562517 T4507 C5556 P50133 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 1562518 T4507 C5556 P50133 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 1562518 T4507 C5556 P50133 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> ASYNC  NEW_CORE C5557 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@a7aaa url=http://127.0.0.1:50145/nodes_used_collection_shard2_replica1 node=127.0.0.1:50145_ C5557_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=active, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50145_, base_url=http://127.0.0.1:50145, leader=true}
[junit4:junit4]   2> 1562609 T4444 C5557 P50145 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 1562625 T4472 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1562670 T4473 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1562679 T4473 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1562683 T4473 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1562776 T4490 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1562786 T4490 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1562790 T4490 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> ASYNC  NEW_CORE C5558 name=nodes_used_collection_shard1_replica1 org.apache.solr.core.SolrCore@10402f2 url=http://127.0.0.1:50154/nodes_used_collection_shard1_replica1 node=127.0.0.1:50154_ C5558_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica1 props:{shard=shard1, state=active, core=nodes_used_collection_shard1_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50154_, base_url=http://127.0.0.1:50154, leader=true}
[junit4:junit4]   2> 1562792 T4460 C5558 P50154 oasc.SolrCore.execute [nodes_used_collection_shard1_replica1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=182 
[junit4:junit4]   2> ASYNC  NEW_CORE C5559 name=nodes_used_collection_shard1_replica2 org.apache.solr.core.SolrCore@12f7054 url=http://127.0.0.1:50133/nodes_used_collection_shard1_replica2 node=127.0.0.1:50133_ C5559_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica2 props:{shard=shard1, state=recovering, core=nodes_used_collection_shard1_replica2, collection=nodes_used_collection, node_name=127.0.0.1:50133_, base_url=http://127.0.0.1:50133}
[junit4:junit4]   2> 1562793 T4507 C5559 P50133 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 1562794 T4507 C5559 P50133 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369747941658\nodes_used_collection_shard1_replica2\data\index.20130528053242000
[junit4:junit4]   2> 1562794 T4507 C5559 P50133 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@141f3d6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1e22347) fullCopy=false
[junit4:junit4]   2> 1562804 T4461 C5558 P50154 oasc.SolrCore.execute [nodes_used_collection_shard1_replica1] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=1 
[junit4:junit4]   2> ASYNC  NEW_CORE C5560 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@a7aaa url=http://127.0.0.1:50145/nodes_used_collection_shard2_replica1 node=127.0.0.1:50145_ C5560_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=active, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50145_, base_url=http://127.0.0.1:50145, leader=true}
[junit4:junit4]   2> 1562806 T4444 C5560 P50145 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1f33676 lockFactory=org.apache.lucene.store.NativeFSLockFactory@bbd508),segFN=segments_1,generation=1}
[junit4:junit4]   2> 1562810 T4444 C5560 P50145 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 1562812 T4444 C5560 P50145 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1f33676 lockFactory=org.apache.lucene.store.NativeFSLockFactory@bbd508),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1f33676 lockFactory=org.apache.lucene.store.NativeFSLockFactory@bbd508),segFN=segments_2,generation=2}
[junit4:junit4]   2> 1562812 T4444 C5560 P50145 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 1562812 T4507 C5559 P50133 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 1562814 T4444 C5560 P50145 oass.SolrIndexSearcher.<init> Opening Searcher@1d83417 realtime
[junit4:junit4]   2> 1562814 T4444 C5560 P50145 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> ASYNC  NEW_CORE C5561 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@a7aaa url=http://127.0.0.1:50145/nodes_used_collection_shard2_replica1 node=127.0.0.1:50145_ C5561_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=active, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50145_, base_url=http://127.0.0.1:50145, leader=true}
[junit4:junit4]   2> 1562814 T4444 C5561 P50145 oasup.LogUpdateProcessor.finish [nodes_used_collection_shard2_replica1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 205
[junit4:junit4]   2> ASYNC  NEW_CORE C5562 name=nodes_used_collection_shard2_replica2 org.apache.solr.core.SolrCore@1b30d8e url=http://127.0.0.1:50172/nodes_used_collection_shard2_replica2 node=127.0.0.1:50172_ C5562_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica2 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica2, collection=nodes_used_collection, node_name=127.0.0.1:50172_, base_url=http://127.0.0.1:50172}
[junit4:junit4]   2> 1562848 T4506 C5562 P50172 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 1562848 T4506 C5562 P50172 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> ASYNC  NEW_CORE C5563 name=nodes_used_collection_shard1_replica2 org.apache.solr.core.SolrCore@12f7054 url=http://127.0.0.1:50133/nodes_used_collection_shard1_replica2 node=127.0.0.1:50133_ C5563_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica2 props:{shard=shard1, state=recovering, core=nodes_used_collection_shard1_replica2, collection=nodes_used_collection, node_name=127.0.0.1:50133_, base_url=http://127.0.0.1:50133}
[junit4:junit4]   2> 1562846 T4507 C5563 P50133 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 1562849 T4507 C5563 P50133 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=nodes_used_collection_shard1_replica2
[junit4:junit4]   2> ASYNC  NEW_CORE C5564 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@a7aaa url=http://127.0.0.1:50145/nodes_used_collection_shard2_replica1 node=127.0.0.1:50145_ C5564_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=active, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50145_, base_url=http://127.0.0.1:50145, leader=true}
[junit4:junit4]   2> 1562820 T4445 C5564 P50145 oasc.SolrCore.execute [nodes_used_collection_shard2_replica1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1 
[junit4:junit4]   2> 1562956 T4527 C5564 P50145 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 1562960 T4527 C5564 P50145 oasc.SolrCore.execute [nodes_used_collection_shard2_replica1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=5 
[junit4:junit4]   2> ASYNC  NEW_CORE C5565 name=nodes_used_collection_shard2_replica2 org.apache.solr.core.SolrCore@1b30d8e url=http://127.0.0.1:50172/nodes_used_collection_shard2_replica2 node=127.0.0.1:50172_ C5565_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica2 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica2, collection=nodes_used_collection, node_name=127.0.0.1:50172_, base_url=http://127.0.0.1:50172}
[junit4:junit4]   2> 1562960 T4506 C5565 P50172 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 1562960 T4506 C5565 P50172 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 1562960 T4506 C5565 P50172 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> ASYNC  NEW_CORE C5566 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@a7aaa url=http://127.0.0.1:50145/nodes_used_collection_shard2_replica1 node=127.0.0.1:50145_ C5566_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=active, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50145_, base_url=http://127.0.0.1:50145, leader=true}
[junit4:junit4]   2> 1563061 T4528 C5566 P50145 oasc.SolrCore.execute [nodes_used_collection_shard2_replica1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=35 
[junit4:junit4]   2> ASYNC  NEW_CORE C5567 name=nodes_used_collection_shard2_replica2 org.apache.solr.core.SolrCore@1b30d8e url=http://127.0.0.1:50172/nodes_used_collection_shard2_replica2 node=127.0.0.1:50172_ C5567_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica2 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica2, collection=nodes_used_collection, node_name=127.0.0.1:50172_, base_url=http://127.0.0.1:50172}
[junit4:junit4]   2> 1563063 T4506 C5567 P50172 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 1563064 T4506 C5567 P50172 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\nodes_used_collection_shard2_replica2\data\index.20130528053242270
[junit4:junit4]   2> 1563064 T4506 C5567 P50172 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@6417cc lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f602ef) fullCopy=false
[junit4:junit4]   2> 1563093 T4445 C5566 P50145 oasc.SolrCore.execute [nodes_used_collection_shard2_replica1] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> ASYNC  NEW_CORE C5568 name=nodes_used_collection_shard1_replica2 org.apache.solr.core.SolrCore@12f7054 url=http://127.0.0.1:50133/nodes_used_collection_shard1_replica2 node=127.0.0.1:50133_ C5568_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica2 props:{shard=shard1, state=recovering, core=nodes_used_collection_shard1_replica2, collection=nodes_used_collection, node_name=127.0.0.1:50133_, base_url=http://127.0.0.1:50133}
[junit4:junit4]   2> 1563172 T4507 C5568 P50133 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@f527d6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@10d8995),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@f527d6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@10d8995),segFN=segments_2,generation=2}
[junit4:junit4]   2> 1563173 T4507 C5568 P50133 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 1563173 T4507 C5568 P50133 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 1563173 T4507 C5568 P50133 oass.SolrIndexSearcher.<init> Opening Searcher@1e88691 main
[junit4:junit4]   2> ASYNC  NEW_CORE C5569 name=nodes_used_collection_shard2_replica2 org.apache.solr.core.SolrCore@1b30d8e url=http://127.0.0.1:50172/nodes_used_collection_shard2_replica2 node=127.0.0.1:50172_ C5569_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica2 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica2, collection=nodes_used_collection, node_name=127.0.0.1:50172_, base_url=http://127.0.0.1:50172}
[junit4:junit4]   2> 1563175 T4506 C5569 P50172 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 1563175 T4503 oasc.SolrCore.registerSearcher [nodes_used_collection_shard1_replica2] Registered new searcher Searcher@1e88691 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 1563176 T4507 C5568 P50133 oasc.CachingDirectoryFactory.closeCacheValue looking to close C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369747941658\nodes_used_collection_shard1_replica2\data\index.20130528053242000 [CachedDir<<refCount=0;path=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369747941658\nodes_used_collection_shard1_replica2\data\index.20130528053242000;done=true>>]
[junit4:junit4]   2> 1563176 T4507 C5568 P50133 oasc.CachingDirectoryFactory.close Closing directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369747941658\nodes_used_collection_shard1_replica2\data\index.20130528053242000
[junit4:junit4]   2> 1563177 T4507 C5568 P50133 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369747941658\nodes_used_collection_shard1_replica2\data\index.20130528053242000
[junit4:junit4]   2> 1563177 T4507 C5568 P50133 oasc.RecoveryStrategy.replay No replay needed. core=nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 1563177 T4507 C5568 P50133 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 1563177 T4507 C5568 P50133 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica2 state=active
[junit4:junit4]   2> 1563181 T4506 C5569 P50172 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 1563181 T4506 C5569 P50172 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=nodes_used_collection_shard2_replica2
[junit4:junit4]   2> 1563232 T4448 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1563240 T4448 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1563244 T4448 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> ASYNC  NEW_CORE C5570 name=nodes_used_collection_shard1_replica2 org.apache.solr.core.SolrCore@12f7054 url=http://127.0.0.1:50133/nodes_used_collection_shard1_replica2 node=127.0.0.1:50133_ C5570_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica2 props:{shard=shard1, state=recovering, core=nodes_used_collection_shard1_replica2, collection=nodes_used_collection, node_name=127.0.0.1:50133_, base_url=http://127.0.0.1:50133}
[junit4:junit4]   2> 1563289 T4507 C5570 P50133 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=nodes_used_collection_shard1_replica2
[junit4:junit4]   2> ASYNC  NEW_CORE C5571 name=nodes_used_collection_shard2_replica2 org.apache.solr.core.SolrCore@1b30d8e url=http://127.0.0.1:50172/nodes_used_collection_shard2_replica2 node=127.0.0.1:50172_ C5571_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica2 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica2, collection=nodes_used_collection, node_name=127.0.0.1:50172_, base_url=http://127.0.0.1:50172}
[junit4:junit4]   2> 1563305 T4506 C5571 P50172 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@838e14 lockFactory=org.apache.lucene.store.NativeFSLockFactory@19df105),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@838e14 lockFactory=org.apache.lucene.store.NativeFSLockFactory@19df105),segFN=segments_2,generation=2}
[junit4:junit4]   2> 1563432 T4506 C5571 P50172 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 1563432 T4506 C5571 P50172 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 1563471 T4506 C5571 P50172 oass.SolrIndexSearcher.<init> Opening Searcher@1bf3c8b main
[junit4:junit4]   2> 1563553 T4502 oasc.SolrCore.registerSearcher [nodes_used_collection_shard2_replica2] Registered new searcher Searcher@1bf3c8b main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 1563579 T4462 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1563588 T4462 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1563593 T4462 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> ASYNC  NEW_CORE C5572 name=nodes_used_collection_shard2_replica2 org.apache.solr.core.SolrCore@1b30d8e url=http://127.0.0.1:50172/nodes_used_collection_shard2_replica2 node=127.0.0.1:50172_ C5572_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica2 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica2, collection=nodes_used_collection, node_name=127.0.0.1:50172_, base_url=http://127.0.0.1:50172}
[junit4:junit4]   2> 1563612 T4506 C5572 P50172 oasc.CachingDirectoryFactory.closeCacheValue looking to close C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\nodes_used_collection_shard2_replica2\data\index.20130528053242270 [CachedDir<<refCount=0;path=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\nodes_used_collection_shard2_replica2\data\index.20130528053242270;done=true>>]
[junit4:junit4]   2> 1563612 T4506 C5572 P50172 oasc.CachingDirectoryFactory.close Closing directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\nodes_used_collection_shard2_replica2\data\index.20130528053242270
[junit4:junit4]   2> ASYNC  NEW_CORE C5573 name=nodes_used_collection_shard2_replica2 org.apache.solr.core.SolrCore@1b30d8e url=http://127.0.0.1:50172/nodes_used_collection_shard2_replica2 node=127.0.0.1:50172_ C5573_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica2 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica2, collection=nodes_used_collection, node_name=127.0.0.1:50172_, base_url=http://127.0.0.1:50172}
[junit4:junit4]   2> 1563662 T4506 C5573 P50172 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\nodes_used_collection_shard2_replica2\data\index.20130528053242270
[junit4:junit4]   2> 1563663 T4506 C5573 P50172 oasc.RecoveryStrategy.replay No replay needed. core=nodes_used_collection_shard2_replica2
[junit4:junit4]   2> 1563663 T4506 C5573 P50172 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=nodes_used_collection_shard2_replica2
[junit4:junit4]   2> 1563663 T4506 C5573 P50172 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica2 state=active
[junit4:junit4]   2> 1563672 T4506 C5573 P50172 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=nodes_used_collection_shard2_replica2
[junit4:junit4]   2> 1563805 T4487 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1563812 T4487 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1563817 T4487 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1563840 T4459 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1563849 T4459 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1563853 T4459 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1563870 T4429 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1563888 T4429 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1563892 T4472 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1564002 T4473 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1564007 T4472 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1564043 T4427 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1564051 T4427 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1564057 T4427 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1564086 T4472 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1564141 T4429 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1564167 T4430 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1564185 T4430 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1564191 T4430 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1564228 T4435 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 1564243 T4472 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1564243 T4472 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1564244 T4472 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1564246 T4435 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:50133__nodes_used_collection_shard1_replica2",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard1_replica2",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:50133_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:50133"}
[junit4:junit4]   2> 1564253 T4473 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1564479 T4472 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1564480 T4472 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1564481 T4472 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1564481 T4472 oasc.SolrCore.<init> [awholynewcollection_0_shard1_replica4] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369747946874\awholynewcollection_0_shard1_replica4\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369747946874\awholynewcollection_0_shard1_replica4\data\
[junit4:junit4]   2> 1564481 T4472 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6909db
[junit4:junit4]   2> 1564482 T4472 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1564482 T4472 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369747946874\awholynewcollection_0_shard1_replica4\data\
[junit4:junit4]   2> 1564483 T4472 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369747946874\awholynewcollection_0_shard1_replica4\data\index/
[junit4:junit4]   2> 1564483 T4472 oasc.SolrCore.initIndex WARN [awholynewcollection_0_shard1_replica4] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369747946874\awholynewcollection_0_shard1_replica4\data\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1564531 T4458 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1564540 T4458 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1564545 T4458 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1564569 T4474 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1564588 T4472 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369747946874\awholynewcollection_0_shard1_replica4\data\index
[junit4:junit4]   2> 1564588 T4474 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1564597 T4474 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1564613 T4490 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1564629 T4425 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1564637 T4462 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1564644 T4473 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1564644 T4490 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1564650 T4425 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1564651 T4443 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1564653 T4425 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1564659 T4443 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1564663 T4443 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1564699 T4447 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1564707 T4447 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1564725 T4447 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1564731 T4473 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1564732 T4473 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1564732 T4473 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1564733 T4435 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:50172__nodes_used_collection_shard2_replica2",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard2_replica2",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:50172_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:50172"}
[junit4:junit4]   2> 1564734 T4459 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1564745 T4462 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1564753 T4490 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1564754 T4448 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1564755 T4472 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1e644a lockFactory=org.apache.lucene.store.NativeFSLockFactory@7af54d),segFN=segments_1,generation=1}
[junit4:junit4]   2> 1564756 T4472 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 1564758 T4473 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1564759 T4473 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1564759 T4473 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1564760 T4473 oasc.SolrCore.<init> [awholynewcollection_0_shard3_replica1] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369747946874\awholynewcollection_0_shard3_replica1\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369747946874\awholynewcollection_0_shard3_replica1\data\
[junit4:junit4]   2> 1564761 T4473 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6909db
[junit4:junit4]   2> 1564761 T4473 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1564762 T4473 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369747946874\awholynewcollection_0_shard3_replica1\data\
[junit4:junit4]   2> 1564767 T4490 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1564767 T4490 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1564767 T4490 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1564768 T4473 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369747946874\awholynewcollection_0_shard3_replica1\data\index/
[junit4:junit4]   2> 1564769 T4473 oasc.SolrCore.initIndex WARN [awholynewcollection_0_shard3_replica1] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369747946874\awholynewcollection_0_shard3_replica1\data\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1564769 T4473 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369747946874\awholynewcollection_0_shard3_replica1\data\index
[junit4:junit4]   2> 1564774 T4472 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1564775 T4472 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1564775 T4472 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1564776 T4472 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1564776 T4472 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1564777 T4472 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1564778 T4472 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1564778 T4472 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1564778 T4472 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1564781 T4490 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1564782 T4490 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1564782 T4490 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1564783 T4490 oasc.SolrCore.<init> [awholynewcollection_0_shard1_replica2] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\awholynewcollection_0_shard1_replica2\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\awholynewcollection_0_shard1_replica2\data\
[junit4:junit4]   2> 1564784 T4490 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6909db
[junit4:junit4]   2> 1564784 T4490 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1564785 T4490 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\awholynewcollection_0_shard1_replica2\data\
[junit4:junit4]   2> 1564787 T4427 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1564795 T4490 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\awholynewcollection_0_shard1_replica2\data\index/
[junit4:junit4]   2> 1564795 T4490 oasc.SolrCore.initIndex WARN [awholynewcollection_0_shard1_replica2] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\awholynewcollection_0_shard1_replica2\data\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1564797 T4490 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369747948786\awholynewcollection_0_shard1_replica2\data\index
[junit4:junit4]   2> 1564802 T4430 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1564810 T4480 oascc.ZkStateReader$2.process A cluster stat

[...truncated too long message...]

   1>        /solr/collections/awholynewcollection_1/leader_elect/shard3/election/89767801087655946-127.0.0.1:50163__awholynewcollection_1_shard3_replica2-n_0000000000 (0)
[junit4:junit4]   1>        /solr/collections/awholynewcollection_1/leader_elect/shard3/election/89767801087655944-127.0.0.1:50154__awholynewcollection_1_shard3_replica1-n_0000000001 (0)
[junit4:junit4]   1>     /solr/collections/awholynewcollection_1/leaders (4)
[junit4:junit4]   1>      /solr/collections/awholynewcollection_1/leaders/shard1 (0)
[junit4:junit4]   1>      DATA:
[junit4:junit4]   1>          {
[junit4:junit4]   1>            "core":"awholynewcollection_1_shard1_replica2",
[junit4:junit4]   1>            "node_name":"127.0.0.1:50172_",
[junit4:junit4]   1>            "base_url":"http://127.0.0.1:50172"}
[junit4:junit4]   1>      /solr/collections/awholynewcollection_1/leaders/shard4 (0)
[junit4:junit4]   1>      DATA:
[junit4:junit4]   1>          {
[junit4:junit4]   1>            "core":"awholynewcollection_1_shard4_replica1",
[junit4:junit4]   1>            "node_name":"127.0.0.1:50172_",
[junit4:junit4]   1>            "base_url":"http://127.0.0.1:50172"}
[junit4:junit4]   1>      /solr/collections/awholynewcollection_1/leaders/shard2 (0)
[junit4:junit4]   1>      DATA:
[junit4:junit4]   1>          {
[junit4:junit4]   1>            "core":"awholynewcollection_1_shard2_replica2",
[junit4:junit4]   1>            "node_name":"127.0.0.1:50145_",
[junit4:junit4]   1>            "base_url":"http://127.0.0.1:50145"}
[junit4:junit4]   1>      /solr/collections/awholynewcollection_1/leaders/shard3 (0)
[junit4:junit4]   1>      DATA:
[junit4:junit4]   1>          {
[junit4:junit4]   1>            "core":"awholynewcollection_1_shard3_replica2",
[junit4:junit4]   1>            "node_name":"127.0.0.1:50163_",
[junit4:junit4]   1>            "base_url":"http://127.0.0.1:50163"}
[junit4:junit4]   1>   /solr/overseer_elect (2)
[junit4:junit4]   1>    /solr/overseer_elect/election (5)
[junit4:junit4]   1>     /solr/overseer_elect/election/89767801087655948-127.0.0.1:50172_-n_0000000004 (0)
[junit4:junit4]   1>     /solr/overseer_elect/election/89767801087655942-127.0.0.1:50145_-n_0000000001 (0)
[junit4:junit4]   1>     /solr/overseer_elect/election/89767801087655939-127.0.0.1:50133_-n_0000000000 (0)
[junit4:junit4]   1>     /solr/overseer_elect/election/89767801087655946-127.0.0.1:50163_-n_0000000003 (0)
[junit4:junit4]   1>     /solr/overseer_elect/election/89767801087655944-127.0.0.1:50154_-n_0000000002 (0)
[junit4:junit4]   1>    /solr/overseer_elect/leader (0)
[junit4:junit4]   1>    DATA:
[junit4:junit4]   1>        {"id":"89767801087655939-127.0.0.1:50133_-n_0000000000"}
[junit4:junit4]   1>  /zookeeper (1)
[junit4:junit4]   1>  DATA:
[junit4:junit4]   1>      
[junit4:junit4]   1> 
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=CollectionsAPIDistributedZkTest -Dtests.method=testDistribSearch -Dtests.seed=C4186F94A266BA7E -Dtests.slow=true -Dtests.locale=sl_SI -Dtests.timezone=America/Nome -Dtests.file.encoding=US-ASCII
[junit4:junit4] FAILURE  317s | CollectionsAPIDistributedZkTest.testDistribSearch <<<
[junit4:junit4]    > Throwable #1: java.lang.AssertionError: There are still nodes recoverying - waited for 230 seconds
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([C4186F94A266BA7E:45FEE18CD539DA42]:0)
[junit4:junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:173)
[junit4:junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131)
[junit4:junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:126)
[junit4:junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:512)
[junit4:junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:146)
[junit4:junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 1859562 T4412 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> NOTE: test params are: codec=Lucene42: {timestamp=MockFixedIntBlock(blockSize=757), range_facet_l=MockFixedIntBlock(blockSize=757), id=Lucene41(blocksize=128), range_facet_sl=PostingsFormat(name=Memory doPackFST= false), a_t=MockFixedIntBlock(blockSize=757), text=PostingsFormat(name=Memory doPackFST= false), range_facet_si=PostingsFormat(name=NestedPulsing), _version_=MockFixedIntBlock(blockSize=757), other_tl1=MockFixedIntBlock(blockSize=757), multiDefault=PostingsFormat(name=NestedPulsing), a_si=PostingsFormat(name=NestedPulsing), intDefault=MockFixedIntBlock(blockSize=757)}, docValues:{timestamp=DocValuesFormat(name=CheapBastard)}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=sl_SI, timezone=America/Nome
[junit4:junit4]   2> NOTE: Windows 7 6.1 x86/Oracle Corporation 1.7.0_21 (32-bit)/cpus=2,threads=2,free=56137744,total=216023040
[junit4:junit4]   2> NOTE: All tests run in this JVM: [FieldMutatingUpdateProcessorTest, XmlUpdateRequestHandlerTest, ZkCLITest, UniqFieldsUpdateProcessorFactoryTest, SignatureUpdateProcessorFactoryTest, QueryEqualityTest, SuggesterTest, TestLMJelinekMercerSimilarityFactory, MultiTermTest, TestSystemIdResolver, TestPostingsSolrHighlighter, StatelessScriptUpdateProcessorFactoryTest, TestPhraseSuggestions, OpenCloseCoreStressTest, SolrPluginUtilsTest, CoreAdminHandlerTest, CachingDirectoryFactoryTest, DebugComponentTest, CurrencyFieldOpenExchangeTest, TestUpdate, TestCoreContainer, TestPropInject, TestQuerySenderListener, DOMUtilTest, ShardRoutingTest, TestSolrQueryParserDefaultOperatorResource, TestDefaultSearchFieldResource, PluginInfoTest, PreAnalyzedFieldTest, TestSolrXMLSerializer, BadComponentTest, TestFieldTypeCollectionResource, SpellPossibilityIteratorTest, ChaosMonkeySafeLeaderTest, TestRTGBase, TestCloudManagedSchema, TestMergePolicyConfig, OutputWriterTest, CircularListTest, TestPseudoReturnFields, SpatialFilterTest, SuggesterWFSTTest, TestCoreDiscovery, FastVectorHighlighterTest, TestFieldResource, TestFunctionQuery, TestFieldCollectionResource, TestDynamicFieldCollectionResource, PathHierarchyTokenizerFactoryTest, TestQuerySenderNoQuery, TestValueSourceCache, TestStressRecovery, SolrIndexConfigTest, SpellingQueryConverterTest, TestXIncludeConfig, NoCacheHeaderTest, SliceStateUpdateTest, BinaryUpdateRequestHandlerTest, TestHashPartitioner, TestLRUCache, ReturnFieldsTest, TestNumberUtils, TestOmitPositions, FileBasedSpellCheckerTest, TestSolrCoreProperties, UnloadDistributedZkTest, TestPropInjectDefaults, RegexBoostProcessorTest, SpellCheckComponentTest, SynonymTokenizerTest, BadCopyFieldTest, IndexBasedSpellCheckerTest, BasicDistributedZkTest, TestJmxIntegration, IndexSchemaTest, TestDefaultSimilarityFactory, TestSolrDeletionPolicy1, TestSweetSpotSimilarityFactory, ScriptEngineTest, SolrInfoMBeanTest, TestCopyFieldCollectionResource, UUIDFieldTest, DocValuesMultiTest, TestExtendedDismaxParser, TestJmxMonitoredMap, TestQueryTypes, QueryElevationComponentTest, DirectUpdateHandlerOptimizeTest, PeerSyncTest, TestCSVResponseWriter, SuggesterTSTTest, AliasIntegrationTest, TestSurroundQueryParser, SortByFunctionTest, TestStressReorder, QueryParsingTest, TestAtomicUpdateErrorCases, ZkNodePropsTest, TestRandomDVFaceting, SpellCheckCollatorTest, SOLR749Test, LeaderElectionIntegrationTest, TestPHPSerializedResponseWriter, TestDFRSimilarityFactory, TestPartialUpdateDeduplication, ChaosMonkeyShardSplitTest, ZkSolrClientTest, SolrCoreCheckLockOnStartupTest, TestReversedWildcardFilterFactory, IndexReaderFactoryTest, TestCloudManagedSchemaAddField, CopyFieldTest, TestFastLRUCache, BasicFunctionalityTest, PrimUtilsTest, TestLazyCores, DistributedQueryElevationComponentTest, DefaultValueUpdateProcessorTest, ChaosMonkeyNothingIsSafeTest, OverseerCollectionProcessorTest, BasicDistributedZk2Test, TestMaxScoreQueryParser, PingRequestHandlerTest, TestReplicationHandler, AutoCommitTest, SampleTest, TestElisionMultitermQuery, FullSolrCloudDistribCmdsTest, TermsComponentTest, TestSolrDeletionPolicy2, DistanceFunctionTest, TestFaceting, TermVectorComponentTest, TestDistributedSearch, TestJoin, CurrencyFieldXmlFileTest, TestSchemaVersionResource, TestCharFilters, PreAnalyzedUpdateProcessorTest, CollectionsAPIDistributedZkTest]
[junit4:junit4] Completed in 317.44s, 1 test, 1 failure <<< FAILURES!

[...truncated 458 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:380: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:360: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:39: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build.xml:181: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\common-build.xml:437: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1240: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:884: There were test failures: 295 suites, 1232 tests, 1 failure, 18 ignored (12 assumptions)

Total time: 68 minutes 53 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.7.0_21 -client -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure