You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2013/06/28 08:15:28 UTC

[JENKINS] Lucene-Solr-4.x-Windows (64bit/jdk1.6.0_45) - Build # 2938 - Still Failing!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Windows/2938/
Java: 64bit/jdk1.6.0_45 -XX:-UseCompressedOops -XX:+UseSerialGC

3 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest

Error Message:
ERROR: SolrIndexSearcher opens=35 closes=33

Stack Trace:
java.lang.AssertionError: ERROR: SolrIndexSearcher opens=35 closes=33
	at __randomizedtesting.SeedInfo.seed([DC839F0952EE3A1E]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:257)
	at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:104)
	at sun.reflect.GeneratedMethodAccessor37.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:700)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:662)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest:     1) Thread[id=1841, name=searcherExecutor-1322-thread-1, state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:957)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:917)         at java.lang.Thread.run(Thread.java:662)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest: 
   1) Thread[id=1841, name=searcherExecutor-1322-thread-1, state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:957)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:917)
        at java.lang.Thread.run(Thread.java:662)
	at __randomizedtesting.SeedInfo.seed([DC839F0952EE3A1E]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=1841, name=searcherExecutor-1322-thread-1, state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:957)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:917)         at java.lang.Thread.run(Thread.java:662)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=1841, name=searcherExecutor-1322-thread-1, state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:957)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:917)
        at java.lang.Thread.run(Thread.java:662)
	at __randomizedtesting.SeedInfo.seed([DC839F0952EE3A1E]:0)




Build Log:
[...truncated 8781 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest
[junit4:junit4]   2> 762430 T1781 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /_yw/iu
[junit4:junit4]   2> 762435 T1781 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\solrtest-ChaosMonkeyNothingIsSafeTest-1372399518728
[junit4:junit4]   2> 762437 T1781 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 762437 T1782 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 762538 T1781 oasc.ZkTestServer.run start zk server on port:63014
[junit4:junit4]   2> 762541 T1781 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 762555 T1788 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3777b395 name:ZooKeeperConnection Watcher:127.0.0.1:63014 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 762555 T1781 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 762555 T1781 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 762570 T1783 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13f8962c4120000, likely client has closed socket
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> 
[junit4:junit4]   2> 762573 T1781 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 762591 T1790 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7713ea6e name:ZooKeeperConnection Watcher:127.0.0.1:63014/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 762591 T1781 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 762591 T1781 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 762605 T1781 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 762614 T1781 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 762625 T1781 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 762636 T1781 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test-files\solr\collection1\conf\solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 762636 T1781 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 762647 T1781 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test-files\solr\collection1\conf\schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 762648 T1781 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 762654 T1781 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test-files\solr\collection1\conf\stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 762655 T1781 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 762660 T1781 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test-files\solr\collection1\conf\protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 762661 T1781 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 762666 T1781 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test-files\solr\collection1\conf\currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 762666 T1781 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 762674 T1781 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test-files\solr\collection1\conf\open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 762674 T1781 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 762680 T1781 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test-files\solr\collection1\conf\mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 762680 T1781 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 762684 T1781 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test-files\solr\collection1\conf\old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 762685 T1781 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 762690 T1781 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test-files\solr\collection1\conf\synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 762691 T1781 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 762698 T1783 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13f8962c4120001, likely client has closed socket
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> 
[junit4:junit4]   2> 763075 T1781 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 763079 T1781 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:63021
[junit4:junit4]   2> 763079 T1781 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 763080 T1781 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 763080 T1781 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1372399518989
[junit4:junit4]   2> 763080 T1781 oasc.CoreContainer$Initializer.initialize looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1372399518989\solr.xml
[junit4:junit4]   2> 763080 T1781 oasc.CoreContainer.<init> New CoreContainer 1717949087
[junit4:junit4]   2> 763081 T1781 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1372399518989\'
[junit4:junit4]   2> 763081 T1781 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1372399518989\'
[junit4:junit4]   2> 763176 T1781 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 763176 T1781 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 763176 T1781 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 763176 T1781 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 763177 T1781 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 763177 T1781 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 763177 T1781 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 763178 T1781 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 763178 T1781 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 763179 T1781 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 763186 T1781 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 763187 T1781 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:63014/solr
[junit4:junit4]   2> 763187 T1781 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 763188 T1781 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 763212 T1801 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5f56966 name:ZooKeeperConnection Watcher:127.0.0.1:63014 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 763213 T1781 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 763219 T1781 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 763220 T1783 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13f8962c4120002, likely client has closed socket
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> 
[junit4:junit4]   2> 763232 T1781 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 763239 T1803 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5bb039c3 name:ZooKeeperConnection Watcher:127.0.0.1:63014/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 763239 T1781 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 763249 T1781 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4]   2> 763263 T1781 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4]   2> 763275 T1781 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 763282 T1781 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63021__yw%2Fiu
[junit4:junit4]   2> 763287 T1781 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:63021__yw%2Fiu
[junit4:junit4]   2> 763298 T1781 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 763321 T1781 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 763330 T1781 oasc.Overseer.start Overseer (id=89941574860013571-127.0.0.1:63021__yw%2Fiu-n_0000000000) starting
[junit4:junit4]   2> 763346 T1781 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4]   2> 763359 T1805 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 763361 T1781 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 763373 T1781 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4]   2> 763381 T1781 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 763389 T1804 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 763401 T1806 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 763401 T1806 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 764905 T1804 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 764907 T1804 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"1",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63021__yw%2Fiu",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63021/_yw/iu"}
[junit4:junit4]   2> 764907 T1804 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 764907 T1804 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4]   2> 764927 T1803 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 765407 T1806 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1372399518989\collection1
[junit4:junit4]   2> 765407 T1806 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 765410 T1806 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 765410 T1806 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 765415 T1806 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1372399518989\collection1\'
[junit4:junit4]   2> 765420 T1806 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-4.x-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1372399518989/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 765422 T1806 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-4.x-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1372399518989/collection1/lib/README' to classloader
[junit4:junit4]   2> 765479 T1806 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 765545 T1806 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 765548 T1806 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 765555 T1806 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 766222 T1806 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4]   2> 766222 T1806 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4]   2> 766222 T1806 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 766232 T1806 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 766236 T1806 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 766256 T1806 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 766263 T1806 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 766267 T1806 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 766268 T1806 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 766268 T1806 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 766269 T1806 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 766270 T1806 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 766270 T1806 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 766270 T1806 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 766271 T1806 oasc.SolrCore.<init> [collection1] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1372399518989\collection1\, dataDir=.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727/control/data\
[junit4:junit4]   2> 766271 T1806 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@900bac2
[junit4:junit4]   2> 766271 T1806 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 766272 T1806 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727\control\data
[junit4:junit4]   2> 766273 T1806 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727/control/data\index/
[junit4:junit4]   2> 766274 T1806 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727\control\data\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 766274 T1806 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727\control\data\index
[junit4:junit4]   2> 766282 T1806 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727\control\data\index,segFN=segments_1,generation=1}
[junit4:junit4]   2> 766282 T1806 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 766285 T1806 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 766285 T1806 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 766286 T1806 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 766286 T1806 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 766287 T1806 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 766287 T1806 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 766287 T1806 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 766288 T1806 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 766288 T1806 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 766295 T1806 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 766311 T1806 oass.SolrIndexSearcher.<init> Opening Searcher@7f077d65 main
[junit4:junit4]   2> 766312 T1806 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 766313 T1806 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 766318 T1807 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7f077d65 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 766322 T1806 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 766322 T1806 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:63021/_yw/iu collection:control_collection shard:shard1
[junit4:junit4]   2> 766323 T1806 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 766340 T1806 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 766347 T1806 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 766347 T1806 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 766347 T1806 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:63021/_yw/iu/collection1/
[junit4:junit4]   2> 766347 T1806 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 766347 T1806 oasc.SyncStrategy.syncToMe http://127.0.0.1:63021/_yw/iu/collection1/ has no replicas
[junit4:junit4]   2> 766347 T1806 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:63021/_yw/iu/collection1/
[junit4:junit4]   2> 766347 T1806 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 766441 T1804 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 766478 T1803 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 766512 T1806 oasc.ZkController.register We are http://127.0.0.1:63021/_yw/iu/collection1/ and leader is http://127.0.0.1:63021/_yw/iu/collection1/
[junit4:junit4]   2> 766512 T1806 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:63021/_yw/iu
[junit4:junit4]   2> 766513 T1806 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 766513 T1806 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 766513 T1806 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 766518 T1806 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 766520 T1781 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0
[junit4:junit4]   2> 766522 T1781 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 766522 T1781 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 766541 T1781 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 766544 T1781 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 766548 T1810 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6a24bbce name:ZooKeeperConnection Watcher:127.0.0.1:63014/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 766548 T1781 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 766550 T1781 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 766553 T1781 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 766946 T1781 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 766948 T1781 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:63033
[junit4:junit4]   2> 766950 T1781 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 766950 T1781 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 766950 T1781 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1372399522846
[junit4:junit4]   2> 766950 T1781 oasc.CoreContainer$Initializer.initialize looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1372399522846\solr.xml
[junit4:junit4]   2> 766950 T1781 oasc.CoreContainer.<init> New CoreContainer 576002109
[junit4:junit4]   2> 766951 T1781 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1372399522846\'
[junit4:junit4]   2> 766951 T1781 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1372399522846\'
[junit4:junit4]   2> 767051 T1781 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 767051 T1781 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 767052 T1781 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 767052 T1781 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 767053 T1781 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 767053 T1781 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 767054 T1781 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 767054 T1781 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 767054 T1781 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 767054 T1781 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 767061 T1781 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 767061 T1781 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:63014/solr
[junit4:junit4]   2> 767062 T1781 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 767064 T1781 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 767083 T1821 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@311bbc88 name:ZooKeeperConnection Watcher:127.0.0.1:63014 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 767083 T1781 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 767087 T1781 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 767094 T1781 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 767098 T1823 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@44a357c5 name:ZooKeeperConnection Watcher:127.0.0.1:63014/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 767098 T1781 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 767107 T1781 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 768001 T1804 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 768003 T1804 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"1",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63021__yw%2Fiu",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63021/_yw/iu"}
[junit4:junit4]   2> 768019 T1810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 768021 T1823 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 768022 T1803 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 768112 T1781 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63033__yw%2Fiu
[junit4:junit4]   2> 768116 T1781 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:63033__yw%2Fiu
[junit4:junit4]   2> 768125 T1823 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 768127 T1803 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 768127 T1803 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 768128 T1810 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 768129 T1810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 768132 T1823 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 768150 T1824 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 768150 T1824 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 769540 T1804 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 769542 T1804 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"2",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63033__yw%2Fiu",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63033/_yw/iu"}
[junit4:junit4]   2> 769543 T1804 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4]   2> 769543 T1804 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4]   2> 769562 T1810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 769564 T1823 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 769564 T1803 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 770156 T1824 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1372399522846\collection1
[junit4:junit4]   2> 770156 T1824 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 770157 T1824 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 770158 T1824 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 770162 T1824 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1372399522846\collection1\'
[junit4:junit4]   2> 770165 T1824 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-4.x-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1372399522846/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 770167 T1824 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-4.x-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1372399522846/collection1/lib/README' to classloader
[junit4:junit4]   2> 770247 T1824 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 770329 T1824 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 770332 T1824 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 770337 T1824 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 771112 T1824 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4]   2> 771112 T1824 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4]   2> 771113 T1824 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 771123 T1824 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 771127 T1824 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 771145 T1824 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 771151 T1824 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 771156 T1824 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 771158 T1824 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 771158 T1824 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 771158 T1824 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 771160 T1824 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 771160 T1824 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 771160 T1824 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 771160 T1824 oasc.SolrCore.<init> [collection1] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1372399522846\collection1\, dataDir=.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727/jetty1\
[junit4:junit4]   2> 771160 T1824 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@900bac2
[junit4:junit4]   2> 771161 T1824 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 771163 T1824 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727\jetty1
[junit4:junit4]   2> 771163 T1824 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727/jetty1\index/
[junit4:junit4]   2> 771164 T1824 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727\jetty1\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 771164 T1824 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727\jetty1\index
[junit4:junit4]   2> 771175 T1824 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727\jetty1\index,segFN=segments_1,generation=1}
[junit4:junit4]   2> 771176 T1824 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 771178 T1824 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 771179 T1824 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 771179 T1824 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 771180 T1824 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 771180 T1824 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 771180 T1824 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 771180 T1824 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 771180 T1824 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 771181 T1824 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 771188 T1824 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 771206 T1824 oass.SolrIndexSearcher.<init> Opening Searcher@79951de6 main
[junit4:junit4]   2> 771208 T1824 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 771208 T1824 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 771215 T1824 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 771215 T1824 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:63033/_yw/iu collection:collection1 shard:shard1
[junit4:junit4]   2> 771216 T1825 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@79951de6 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 771217 T1824 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 771235 T1824 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 771242 T1824 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 771243 T1824 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 771243 T1824 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:63033/_yw/iu/collection1/
[junit4:junit4]   2> 771243 T1824 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 771243 T1824 oasc.SyncStrategy.syncToMe http://127.0.0.1:63033/_yw/iu/collection1/ has no replicas
[junit4:junit4]   2> 771243 T1824 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:63033/_yw/iu/collection1/
[junit4:junit4]   2> 771244 T1824 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 772583 T1804 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 772613 T1810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 772614 T1823 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 772615 T1803 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 772632 T1824 oasc.ZkController.register We are http://127.0.0.1:63033/_yw/iu/collection1/ and leader is http://127.0.0.1:63033/_yw/iu/collection1/
[junit4:junit4]   2> 772632 T1824 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:63033/_yw/iu
[junit4:junit4]   2> 772633 T1824 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 772633 T1824 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 772634 T1824 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 772638 T1824 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 772641 T1781 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0
[junit4:junit4]   2> 772643 T1781 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 772643 T1781 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 773049 T1781 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 773051 T1781 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:63042
[junit4:junit4]   2> 773052 T1781 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 773053 T1781 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 773053 T1781 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1372399528953
[junit4:junit4]   2> 773053 T1781 oasc.CoreContainer$Initializer.initialize looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1372399528953\solr.xml
[junit4:junit4]   2> 773053 T1781 oasc.CoreContainer.<init> New CoreContainer 2018049261
[junit4:junit4]   2> 773053 T1781 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1372399528953\'
[junit4:junit4]   2> 773054 T1781 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1372399528953\'
[junit4:junit4]   2> 773145 T1781 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 773145 T1781 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 773145 T1781 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 773146 T1781 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 773147 T1781 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 773147 T1781 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 773147 T1781 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 773147 T1781 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 773147 T1781 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 773147 T1781 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 773155 T1781 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 773155 T1781 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:63014/solr
[junit4:junit4]   2> 773155 T1781 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 773158 T1781 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 773174 T1837 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13f43e25 name:ZooKeeperConnection Watcher:127.0.0.1:63014 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 773174 T1781 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 773178 T1781 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 773187 T1781 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 773190 T1839 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@bf65c0c name:ZooKeeperConnection Watcher:127.0.0.1:63014/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 773190 T1781 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 773200 T1781 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 774135 T1804 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 774137 T1804 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"2",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63033__yw%2Fiu",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63033/_yw/iu"}
[junit4:junit4]   2> 774159 T1810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 774159 T1803 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 774160 T1823 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 774161 T1839 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 774205 T1781 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63042__yw%2Fiu
[junit4:junit4]   2> 774210 T1781 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:63042__yw%2Fiu
[junit4:junit4]   2> 774219 T1823 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 774221 T1810 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 774222 T1810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 774222 T1803 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 774222 T1803 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 774223 T1839 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 774223 T1839 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 774231 T1823 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 774247 T1840 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 774247 T1840 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 775680 T1804 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 775682 T1804 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"3",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63042__yw%2Fiu",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63042/_yw/iu"}
[junit4:junit4]   2> 775682 T1804 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 775683 T1804 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4]   2> 775707 T1810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 775707 T1803 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 775709 T1823 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 775709 T1839 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 776253 T1840 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1372399528953\collection1
[junit4:junit4]   2> 776253 T1840 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 776255 T1840 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 776255 T1840 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 776261 T1840 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1372399528953\collection1\'
[junit4:junit4]   2> 776264 T1840 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-4.x-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1372399528953/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 776266 T1840 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-4.x-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1372399528953/collection1/lib/README' to classloader
[junit4:junit4]   2> 776347 T1840 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 776412 T1840 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 776414 T1840 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 776420 T1840 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 777105 T1840 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4]   2> 777107 T1840 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4]   2> 777107 T1840 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 777117 T1840 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 777121 T1840 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 777140 T1840 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 777145 T1840 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 777150 T1840 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 777151 T1840 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 777151 T1840 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 777152 T1840 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 777153 T1840 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 777154 T1840 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 777154 T1840 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 777154 T1840 oasc.SolrCore.<init> [collection1] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1372399528953\collection1\, dataDir=.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727/jetty2\
[junit4:junit4]   2> 777154 T1840 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@900bac2
[junit4:junit4]   2> 777155 T1840 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 777156 T1840 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727\jetty2
[junit4:junit4]   2> 777156 T1840 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727/jetty2\index/
[junit4:junit4]   2> 777157 T1840 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727\jetty2\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 777157 T1840 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727\jetty2\index
[junit4:junit4]   2> 777169 T1840 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727\jetty2\index,segFN=segments_1,generation=1}
[junit4:junit4]   2> 777169 T1840 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 777171 T1840 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 777172 T1840 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 777173 T1840 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 777174 T1840 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 777175 T1840 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 777175 T1840 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 777175 T1840 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 777177 T1840 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 777177 T1840 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 777186 T1840 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 777196 T1840 oass.SolrIndexSearcher.<init> Opening Searcher@12cdc6e7 main
[junit4:junit4]   2> 777198 T1840 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 777198 T1840 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 777215 T1841 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@12cdc6e7 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 777219 T1840 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 777219 T1840 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:63042/_yw/iu collection:collection1 shard:shard2
[junit4:junit4]   2> 777221 T1840 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 777235 T1840 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 777240 T1840 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 777240 T1840 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 777240 T1840 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:63042/_yw/iu/collection1/
[junit4:junit4]   2> 777240 T1840 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 777241 T1840 oasc.SyncStrategy.syncToMe http://127.0.0.1:63042/_yw/iu/collection1/ has no replicas
[junit4:junit4]   2> 777241 T1840 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:63042/_yw/iu/collection1/
[junit4:junit4]   2> 777241 T1840 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 778733 T1804 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 778764 T1810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 778764 T1823 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 778765 T1839 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 778764 T1803 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 778778 T1840 oasc.ZkController.register We are http://127.0.0.1:63042/_yw/iu/collection1/ and leader is http://127.0.0.1:63042/_yw/iu/collection1/
[junit4:junit4]   2> 778778 T1840 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:63042/_yw/iu
[junit4:junit4]   2> 778779 T1840 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 778779 T1840 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 778779 T1840 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 778787 T1840 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 778790 T1781 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0
[junit4:junit4]   2> 778791 T1781 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 778792 T1781 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 779197 T1781 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 779201 T1781 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:63051
[junit4:junit4]   2> 779202 T1781 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 779202 T1781 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 779202 T1781 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1372399535102
[junit4:junit4]   2> 779203 T1781 oasc.CoreContainer$Initializer.initialize looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1372399535102\solr.xml
[junit4:junit4]   2> 779203 T1781 oasc.CoreContainer.<init> New CoreContainer 824916335
[junit4:junit4]   2> 779204 T1781 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1372399535102\'
[junit4:junit4]   2> 779204 T1781 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1372399535102\'
[junit4:junit4]   2> 779299 T1781 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 779299 T1781 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 779299 T1781 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 779299 T1781 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 779300 T1781 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 779300 T1781 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 779301 T1781 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 779301 T1781 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 779302 T1781 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 779302 T1781 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 779308 T1781 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 779308 T1781 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:63014/solr
[junit4:junit4]   2> 779308 T1781 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 779311 T1781 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 779326 T1853 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@df5e995 name:ZooKeeperConnection Watcher:127.0.0.1:63014 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 779327 T1781 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 779333 T1781 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 779341 T1781 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 779345 T1855 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5ad1d3b8 name:ZooKeeperConnection Watcher:127.0.0.1:63014/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 779345 T1781 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 779353 T1781 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 780296 T1804 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 780297 T1804 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"3",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63042__yw%2Fiu",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63042/_yw/iu"}
[junit4:junit4]   2> 780312 T1803 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 780312 T1855 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 780312 T1810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 780313 T1823 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 780313 T1839 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 780358 T1781 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63051__yw%2Fiu
[junit4:junit4]   2> 780364 T1781 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:63051__yw%2Fiu
[junit4:junit4]   2> 780372 T1855 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 780373 T1823 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 780375 T1810 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 780375 T1810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 780377 T1803 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 780377 T1803 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 780379 T1839 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 780379 T1839 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 780386 T1855 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 780387 T1823 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 780404 T1856 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 780404 T1856 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 781828 T1804 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 781831 T1804 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"4",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63051__yw%2Fiu",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63051/_yw/iu"}
[junit4:junit4]   2> 781831 T1804 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 781831 T1804 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4]   2> 781852 T1810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 781852 T1803 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 781854 T1855 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 781854 T1839 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 781854 T1823 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 782409 T1856 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1372399535102\collection1
[junit4:junit4]   2> 782409 T1856 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 782411 T1856 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 782412 T1856 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 782416 T1856 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1372399535102\collection1\'
[junit4:junit4]   2> 782418 T1856 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-4.x-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1372399535102/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 782420 T1856 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-4.x-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1372399535102/collection1/lib/README' to classloader
[junit4:junit4]   2> 782487 T1856 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 782554 T1856 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 782555 T1856 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 782560 T1856 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 783282 T1856 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4]   2> 783284 T1856 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4]   2> 783284 T1856 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 783295 T1856 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 783300 T1856 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 783319 T1856 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 783326 T1856 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 783332 T1856 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 783334 T1856 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 783334 T1856 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 783334 T1856 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 783336 T1856 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 783336 T1856 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 783336 T1856 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 783337 T1856 oasc.SolrCore.<init> [collection1] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1372399535102\collection1\, dataDir=.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727/jetty3\
[junit4:junit4]   2> 783337 T1856 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@900bac2
[junit4:junit4]   2> 783337 T1856 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 783339 T1856 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727\jetty3
[junit4:junit4]   2> 783339 T1856 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727/jetty3\index/
[junit4:junit4]   2> 783340 T1856 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727\jetty3\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 783340 T1856 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727\jetty3\index
[junit4:junit4]   2> 783349 T1856 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727\jetty3\index,segFN=segments_1,generation=1}
[junit4:junit4]   2> 783349 T1856 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 783352 T1856 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 783353 T1856 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 783353 T1856 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 783354 T1856 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 783355 T1856 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 783355 T1856 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 783355 T1856 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 783356 T1856 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 783356 T1856 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 783364 T1856 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 783374 T1856 oass.SolrIndexSearcher.<init> Opening Searcher@302e127f main
[junit4:junit4]   2> 783376 T1856 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 783376 T1856 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 783389 T1857 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@302e127f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 783391 T1856 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 783391 T1856 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:63051/_yw/iu collection:collection1 shard:shard1
[junit4:junit4]   2> 783397 T1856 oasc.ZkController.register We are http://127.0.0.1:63051/_yw/iu/collection1/ and leader is http://127.0.0.1:63033/_yw/iu/collection1/
[junit4:junit4]   2> 783397 T1856 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:63051/_yw/iu
[junit4:junit4]   2> 783398 T1856 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 783398 T1856 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C1941 name=collection1 org.apache.solr.core.SolrCore@2c433166 url=http://127.0.0.1:63051/_yw/iu/collection1 node=127.0.0.1:63051__yw%2Fiu C1941_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:63051__yw%2Fiu, base_url=http://127.0.0.1:63051/_yw/iu}
[junit4:junit4]   2> 783399 T1858 C1941 P63051 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 783399 T1858 C1941 P63051 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 783400 T1856 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 783400 T1858 C1941 P63051 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 783400 T1858 C1941 P63051 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 783402 T1781 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0
[junit4:junit4]   2> 783402 T1781 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 783403 T1781 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 783403 T1858 C1941 P63051 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 783413 T1816 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4]   2> 783797 T1781 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 783800 T1781 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:63061
[junit4:junit4]   2> 783801 T1781 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 783801 T1781 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 783802 T1781 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty4-1372399539703
[junit4:junit4]   2> 783802 T1781 oasc.CoreContainer$Initializer.initialize looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty4-1372399539703\solr.xml
[junit4:junit4]   2> 783802 T1781 oasc.CoreContainer.<init> New CoreContainer 983756172
[junit4:junit4]   2> 783803 T1781 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty4-1372399539703\'
[junit4:junit4]   2> 783803 T1781 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty4-1372399539703\'
[junit4:junit4]   2> 783899 T1781 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 783900 T1781 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 783900 T1781 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 783900 T1781 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 783900 T1781 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 783900 T1781 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 783901 T1781 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 783901 T1781 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 783901 T1781 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 783902 T1781 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 783909 T1781 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 783909 T1781 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:63014/solr
[junit4:junit4]   2> 783910 T1781 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 783911 T1781 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 783928 T1870 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@77f1d9e2 name:ZooKeeperConnection Watcher:127.0.0.1:63014 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 783929 T1781 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 783932 T1781 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 783935 T1783 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13f8962c412000b, likely client has closed socket
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> 
[junit4:junit4]   2> 783943 T1781 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 783948 T1872 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@385a8b21 name:ZooKeeperConnection Watcher:127.0.0.1:63014/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 783948 T1781 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 783960 T1781 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 784875 T1804 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 784877 T1804 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"4",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63051__yw%2Fiu",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63051/_yw/iu"}
[junit4:junit4]   2> 784898 T1872 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 784898 T1803 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 784900 T1823 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 784901 T1855 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 784898 T1810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 784900 T1839 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 784966 T1781 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63061__yw%2Fiu
[junit4:junit4]   2> 784972 T1781 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:63061__yw%2Fiu
[junit4:junit4]   2> 784980 T1872 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 784981 T1855 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 784982 T1823 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 784984 T1810 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 784984 T1810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 784985 T1803 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 784985 T1803 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 784987 T1839 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 784988 T1839 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 784993 T1872 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 784996 T1855 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 784997 T1823 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 785014 T1873 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 785015 T1873 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 785414 T1816 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4]   2> 785415 T1816 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=4&state=recovering&nodeName=127.0.0.1:63051__yw%252Fiu&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2001 
[junit4:junit4]   2> 786421 T1804 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 786423 T1804 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"5",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63061__yw%2Fiu",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63061/_yw/iu"}
[junit4:junit4]   2> 786424 T1804 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 786424 T1804 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4]   2> 786441 T1803 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 786442 T1810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 786442 T1823 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 786442 T1839 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 786441 T1872 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 786442 T1855 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 787019 T1873 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty4-1372399539703\collection1
[junit4:junit4]   2> 787019 T1873 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 787021 T1873 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 787023 T1873 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 787028 T1873 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty4-1372399539703\collection1\'
[junit4:junit4]   2> 787031 T1873 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-4.x-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty4-1372399539703/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 787033 T1873 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-4.x-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty4-1372399539703/collection1/lib/README' to classloader
[junit4:junit4]   2> 787100 T1873 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 787166 T1873 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 787169 T1873 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 787174 T1873 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> ASYNC  NEW_CORE C1942 name=collection1 org.apache.solr.core.SolrCore@2c433166 url=http://127.0.0.1:63051/_yw/iu/collection1 node=127.0.0.1:63051__yw%2Fiu C1942_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:63051__yw%2Fiu, base_url=http://127.0.0.1:63051/_yw/iu}
[junit4:junit4]   2> 787416 T1858 C1942 P63051 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:63033/_yw/iu/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 787416 T1858 C1942 P63051 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:63051/_yw/iu START replicas=[http://127.0.0.1:63033/_yw/iu/collection1/] nUpdates=100
[junit4:junit4]   2> 787416 T1858 C1942 P63051 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 787417 T1858 C1942 P63051 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 787417 T1858 C1942 P63051 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 787417 T1858 C1942 P63051 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 787418 T1858 C1942 P63051 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 787418 T1858 C1942 P63051 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:63033/_yw/iu/collection1/. core=collection1
[junit4:junit4]   2> 787418 T1858 C1942 P63051 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C1943 name=collection1 org.apache.solr.core.SolrCore@6fbcf7f2 url=http://127.0.0.1:63033/_yw/iu/collection1 node=127.0.0.1:63033__yw%2Fiu C1943_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:63033__yw%2Fiu, base_url=http://127.0.0.1:63033/_yw/iu, leader=true}
[junit4:junit4]   2> 787431 T1816 C1943 P63033 oasc.SolrCore.execute [collection1] webapp=/_yw/iu path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 787433 T1814 C1943 P63033 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 787440 T1814 C1943 P63033 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   2> 		commit{dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727\jetty1\index,segFN=segments_1,generation=1}
[junit4:junit4]   2> 787441 T1814 C1943 P63033 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 787442 T1814 C1943 P63033 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4]   2> 787442 T1814 C1943 P63033 oass.SolrIndexSearcher.<init> Opening Searcher@3948bc7 realtime
[junit4:junit4]   2> 787442 T1814 C1943 P63033 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 787443 T1814 C1943 P63033 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yw/iu path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4]   2> 787443 T1858 C1942 P63051 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 787443 T1858 C1942 P63051 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 787447 T1815 C1943 P63033 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 787447 T1815 C1943 P63033 oasc.SolrCore.execute [collection1] webapp=/_yw/iu path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 787448 T1858 C1942 P63051 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 787448 T1858 C1942 P63051 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 787448 T1858 C1942 P63051 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 787448 T1858 C1942 P63051 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 787454 T1858 C1942 P63051 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 787894 T1873 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4]   2> 787895 T1873 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4]   2> 787896 T1873 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 787906 T1873 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 787910 T1873 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 787931 T1873 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 787936 T1873 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 787940 T1873 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 787942 T1873 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 787942 T1873 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 787943 T1873 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 787944 T1873 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 787944 T1873 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 787945 T1873 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 787945 T1873 oasc.SolrCore.<init> [collection1] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty4-1372399539703\collection1\, dataDir=.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727/jetty4\
[junit4:junit4]   2> 787945 T1873 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@900bac2
[junit4:junit4]   2> 787946 T1873 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 787947 T1873 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727\jetty4
[junit4:junit4]   2> 787947 T1873 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727/jetty4\index/
[junit4:junit4]   2> 787948 T1873 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727\jetty4\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 787949 T1873 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727\jetty4\index
[junit4:junit4]   2> 787959 T1873 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727\jetty4\index,segFN=segments_1,generation=1}
[junit4:junit4]   2> 787959 T1873 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 787960 T1804 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 787961 T1804 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"4",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63051__yw%2Fiu",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63051/_yw/iu"}
[junit4:junit4]   2> 787963 T1873 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 787964 T1873 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 787964 T1873 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 787966 T1873 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 787967 T1873 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 787967 T1873 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 787970 T1803 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 787971 T1855 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 787971 T1823 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 787978 T1839 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 787979 T1810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 787980 T1873 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 787981 T1873 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 787982 T1872 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 787982 T1873 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 787992 T1873 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 788004 T1873 oass.SolrIndexSearcher.<init> Opening Searcher@76fcfd6d main
[junit4:junit4]   2> 788006 T1873 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 788007 T1873 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 788011 T1875 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@76fcfd6d main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 788013 T1873 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 788013 T1873 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:63061/_yw/iu collection:collection1 shard:shard2
[junit4:junit4]   2> 788018 T1873 oasc.ZkController.register We are http://127.0.0.1:63061/_yw/iu/collection1/ and leader is http://127.0.0.1:63042/_yw/iu/collection1/
[junit4:junit4]   2> 788018 T1873 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:63061/_yw/iu
[junit4:junit4]   2> 788018 T1873 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 788019 T1873 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C1944 name=collection1 org.apache.solr.core.SolrCore@2bbde465 url=http://127.0.0.1:63061/_yw/iu/collection1 node=127.0.0.1:63061__yw%2Fiu C1944_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:63061__yw%2Fiu, base_url=http://127.0.0.1:63061/_yw/iu}
[junit4:junit4]   2> 788019 T1876 C1944 P63061 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 788019 T1876 C1944 P63061 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 788020 T1876 C1944 P63061 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 788020 T1876 C1944 P63061 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 788020 T1873 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 788022 T1876 C1944 P63061 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 788023 T1781 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0
[junit4:junit4]   2> 788023 T1781 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 788024 T1781 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 788032 T1834 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4]   2> 788427 T1781 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 788430 T1781 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:63074
[junit4:junit4]   2> 788430 T1781 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 788430 T1781 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 788430 T1781 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty5-1372399544321
[junit4:junit4]   2> 788430 T1781 oasc.CoreContainer$Initializer.initialize looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty5-1372399544321\solr.xml
[junit4:junit4]   2> 788430 T1781 oasc.CoreContainer.<init> New CoreContainer 1887899001
[junit4:junit4]   2> 788431 T1781 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty5-1372399544321\'
[junit4:junit4]   2> 788431 T1781 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty5-1372399544321\'
[junit4:junit4]   2> 788525 T1781 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 788525 T1781 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 788526 T1781 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 788526 T1781 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 788526 T1781 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 788527 T1781 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 788527 T1781 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 788528 T1781 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 788528 T1781 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 788529 T1781 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 788536 T1781 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 788536 T1781 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:63014/solr
[junit4:junit4]   2> 788536 T1781 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 788538 T1781 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 788553 T1888 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6a245033 name:ZooKeeperConnection Watcher:127.0.0.1:63014 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 788554 T1781 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 788558 T1781 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 788558 T1783 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13f8962c412000d, likely client has closed socket
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> 
[junit4:junit4]   2> 788565 T1781 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 788569 T1890 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@76b0537e name:ZooKeeperConnection Watcher:127.0.0.1:63014/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 788570 T1781 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 788580 T1781 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 789494 T1804 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 789496 T1804 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"5",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63061__yw%2Fiu",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63061/_yw/iu"}
[junit4:junit4]   2> 789513 T1890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 789514 T1810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 789516 T1872 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 789515 T1839 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 789515 T1803 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 789515 T1855 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 789515 T1823 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 789586 T1781 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63074__yw%2Fiu
[junit4:junit4]   2> 789591 T1781 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:63074__yw%2Fiu
[junit4:junit4]   2> 789598 T1872 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 789599 T1855 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 789599 T1823 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 789601 T1890 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 789601 T1890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 789602 T1810 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 789602 T1810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 789606 T1803 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 789606 T1803 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 789608 T1839 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 789610 T1839 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 789617 T1872 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 789617 T1823 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 789617 T1855 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 789637 T1891 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 789637 T1891 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 790032 T1834 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4]   2> 790033 T1834 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=5&state=recovering&nodeName=127.0.0.1:63061__yw%252Fiu&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2001 
[junit4:junit4]   2> 791039 T1804 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 791042 T1804 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"6",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63074__yw%2Fiu",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63074/_yw/iu"}
[junit4:junit4]   2> 791042 T1804 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 791043 T1804 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4]   2> 791068 T1890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 791068 T1810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 791069 T1872 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 791070 T1803 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 791070 T1855 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 791071 T1839 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 791071 T1823 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 791640 T1891 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty5-1372399544321\collection1
[junit4:junit4]   2> 791640 T1891 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 791642 T1891 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 791642 T1891 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 791648 T1891 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty5-1372399544321\collection1\'
[junit4:junit4]   2> 791652 T1891 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-4.x-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty5-1372399544321/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 791653 T1891 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-4.x-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty5-1372399544321/collection1/lib/README' to classloader
[junit4:junit4]   2> 791743 T1891 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 791807 T1891 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 791810 T1891 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 791815 T1891 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> ASYNC  NEW_CORE C1945 name=collection1 org.apache.solr.core.SolrCore@2bbde465 url=http://127.0.0.1:63061/_yw/iu/collection1 node=127.0.0.1:63061__yw%2Fiu C1945_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:63061__yw%2Fiu, base_url=http://127.0.0.1:63061/_yw/iu}
[junit4:junit4]   2> 792043 T1876 C1945 P63061 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:63042/_yw/iu/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 792043 T1876 C1945 P63061 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:63061/_yw/iu START replicas=[http://127.0.0.1:63042/_yw/iu/collection1/] nUpdates=100
[junit4:junit4]   2> 792043 T1876 C1945 P63061 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 792044 T1876 C1945 P63061 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 792044 T1876 C1945 P63061 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 792044 T1876 C1945 P63061 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 792044 T1876 C1945 P63061 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 792045 T1876 C1945 P63061 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:63042/_yw/iu/collection1/. core=collection1
[junit4:junit4]   2> 792045 T1876 C1945 P63061 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C1946 name=collection1 org.apache.solr.core.SolrCore@b0b0eb4 url=http://127.0.0.1:63042/_yw/iu/collection1 node=127.0.0.1:63042__yw%2Fiu C1946_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:63042__yw%2Fiu, base_url=http://127.0.0.1:63042/_yw/iu, leader=true}
[junit4:junit4]   2> 792057 T1833 C1946 P63042 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 792064 T1833 C1946 P63042 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   2> 		commit{dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727\jetty2\index,segFN=segments_1,generation=1}
[junit4:junit4]   2> 792066 T1832 C1946 P63042 oasc.SolrCore.execute [collection1] webapp=/_yw/iu path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 792067 T1833 C1946 P63042 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 792067 T1833 C1946 P63042 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4]   2> 792068 T1833 C1946 P63042 oass.SolrIndexSearcher.<init> Opening Searcher@7c0706c6 realtime
[junit4:junit4]   2> 792068 T1833 C1946 P63042 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 792068 T1833 C1946 P63042 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yw/iu path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 11
[junit4:junit4]   2> 792069 T1876 C1945 P63061 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 792069 T1876 C1945 P63061 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 792072 T1835 C1946 P63042 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 792072 T1835 C1946 P63042 oasc.SolrCore.execute [collection1] webapp=/_yw/iu path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 792072 T1876 C1945 P63061 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 792072 T1876 C1945 P63061 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 792072 T1876 C1945 P63061 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 792072 T1876 C1945 P63061 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 792076 T1876 C1945 P63061 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 792497 T1891 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4]   2> 792497 T1891 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4]   2> 792498 T1891 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 792508 T1891 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 792512 T1891 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 792532 T1891 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 792537 T1891 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 792542 T1891 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 792554 T1891 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 792554 T1891 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 792554 T1891 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 792555 T1891 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 792555 T1891 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 792555 T1891 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 792557 T1891 oasc.SolrCore.<init> [collection1] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty5-1372399544321\collection1\, dataDir=.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727/jetty5\
[junit4:junit4]   2> 792557 T1891 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@900bac2
[junit4:junit4]   2> 792557 T1891 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 792558 T1891 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727\jetty5
[junit4:junit4]   2> 792558 T1891 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727/jetty5\index/
[junit4:junit4]   2> 792559 T1891 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727\jetty5\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 792559 T1891 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727\jetty5\index
[junit4:junit4]   2> 792568 T1891 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727\jetty5\index,segFN=segments_1,generation=1}
[junit4:junit4]   2> 792568 T1891 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 792571 T1891 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 792571 T1891 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 792572 T1891 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 792572 T1891 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 792573 T1891 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 792573 T1891 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 792577 T1891 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 792578 T1891 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 792578 T1891 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 792586 T1891 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 792588 T1804 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 792589 T1804 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"5",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63061__yw%2Fiu",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63061/_yw/iu"}
[junit4:junit4]   2> 792597 T1803 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 792598 T1855 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 792598 T1839 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 792598 T1823 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 792599 T1890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 792599 T1810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 792599 T1872 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 792600 T1891 oass.SolrIndexSearcher.<init> Opening Searcher@251f958e main
[junit4:junit4]   2> 792604 T1891 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 792604 T1891 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 792611 T1893 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@251f958e main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 792614 T1891 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 792615 T1891 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:63074/_yw/iu collection:collection1 shard:shard1
[junit4:junit4]   2> 792620 T1891 oasc.ZkController.register We are http://127.0.0.1:63074/_yw/iu/collection1/ and leader is http://127.0.0.1:63033/_yw/iu/collection1/
[junit4:junit4]   2> 792620 T1891 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:63074/_yw/iu
[junit4:junit4]   2> 792620 T1891 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 792620 T1891 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C1947 name=collection1 org.apache.solr.core.SolrCore@6477b82 url=http://127.0.0.1:63074/_yw/iu/collection1 node=127.0.0.1:63074__yw%2Fiu C1947_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:63074__yw%2Fiu, base_url=http://127.0.0.1:63074/_yw/iu}
[junit4:junit4]   2> 792621 T1894 C1947 P63074 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 792621 T1894 C1947 P63074 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 792621 T1894 C1947 P63074 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 792621 T1894 C1947 P63074 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 792622 T1891 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 792624 T1894 C1947 P63074 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 792625 T1781 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0
[junit4:junit4]   2> 792625 T1781 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 792626 T1781 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 792634 T1819 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 6, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4]   2> 793049 T1781 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 793052 T1781 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:63087
[junit4:junit4]   2> 793053 T1781 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 793053 T1781 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 793054 T1781 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty6-1372399548924
[junit4:junit4]   2> 793054 T1781 oasc.CoreContainer$Initializer.initialize looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty6-1372399548924\solr.xml
[junit4:junit4]   2> 793055 T1781 oasc.CoreContainer.<init> New CoreContainer 464498080
[junit4:junit4]   2> 793055 T1781 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty6-1372399548924\'
[junit4:junit4]   2> 793055 T1781 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty6-1372399548924\'
[junit4:junit4]   2> 793153 T1781 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 793153 T1781 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 793153 T1781 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 793153 T1781 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 793155 T1781 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 793155 T1781 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 793156 T1781 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 793156 T1781 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 793156 T1781 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 793156 T1781 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 793163 T1781 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 793164 T1781 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:63014/solr
[junit4:junit4]   2> 793165 T1781 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 793166 T1781 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 793183 T1906 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@51cffca name:ZooKeeperConnection Watcher:127.0.0.1:63014 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 793184 T1781 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 793188 T1781 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 793188 T1783 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13f8962c412000f, likely client has closed socket
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> 
[junit4:junit4]   2> 793193 T1781 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 793196 T1908 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7b143076 name:ZooKeeperConnection Watcher:127.0.0.1:63014/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 793197 T1781 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 793207 T1781 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 794111 T1804 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 794112 T1804 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"6",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63074__yw%2Fiu",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63074/_yw/iu"}
[junit4:junit4]   2> 794125 T1803 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 794126 T1855 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 794127 T1823 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 794127 T1890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 794128 T1810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 794129 T1872 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 794127 T1908 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 794127 T1839 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 794213 T1781 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63087__yw%2Fiu
[junit4:junit4]   2> 794215 T1781 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:63087__yw%2Fiu
[junit4:junit4]   2> 794219 T1855 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 794220 T1908 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 794220 T1872 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 794220 T1823 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 794222 T1803 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 794222 T1803 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 794222 T1890 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 794222 T1890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 794222 T1810 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 794222 T1810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 794222 T1839 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 794223 T1839 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 794226 T1908 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 794227 T1855 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 794227 T1823 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 794228 T1872 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 794238 T1909 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 794238 T1909 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 794634 T1819 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 6, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4]   2> 794634 T1819 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=6&state=recovering&nodeName=127.0.0.1:63074__yw%252Fiu&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2001 
[junit4:junit4]   2> 795644 T1804 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 795646 T1804 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"7",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63087__yw%2Fiu",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63087/_yw/iu"}
[junit4:junit4]   2> 795646 T1804 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 795647 T1804 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4]   2> 795668 T1803 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 795669 T1855 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 795670 T1823 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 795671 T1839 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 795671 T1908 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 795672 T1890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 795672 T1810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 795673 T1872 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 796240 T1909 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty6-1372399548924\collection1
[junit4:junit4]   2> 796240 T1909 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 796243 T1909 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 796243 T1909 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 796248 T1909 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty6-1372399548924\collection1\'
[junit4:junit4]   2> 796252 T1909 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-4.x-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty6-1372399548924/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 796254 T1909 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-4.x-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty6-1372399548924/collection1/lib/README' to classloader
[junit4:junit4]   2> 796349 T1909 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 796413 T1909 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 796415 T1909 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 796420 T1909 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> ASYNC  NEW_CORE C1948 name=collection1 org.apache.solr.core.SolrCore@6477b82 url=http://127.0.0.1:63074/_yw/iu/collection1 node=127.0.0.1:63074__yw%2Fiu C1948_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:63074__yw%2Fiu, base_url=http://127.0.0.1:63074/_yw/iu}
[junit4:junit4]   2> 796636 T1894 C1948 P63074 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:63033/_yw/iu/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 796636 T1894 C1948 P63074 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:63074/_yw/iu START replicas=[http://127.0.0.1:63033/_yw/iu/collection1/] nUpdates=100
[junit4:junit4]   2> 796637 T1894 C1948 P63074 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 796637 T1894 C1948 P63074 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 796638 T1894 C1948 P63074 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 796638 T1894 C1948 P63074 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 796638 T1894 C1948 P63074 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 796638 T1894 C1948 P63074 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:63033/_yw/iu/collection1/. core=collection1
[junit4:junit4]   2> 796638 T1894 C1948 P63074 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C1949 name=collection1 org.apache.solr.core.SolrCore@6fbcf7f2 url=http://127.0.0.1:63033/_yw/iu/collection1 node=127.0.0.1:63033__yw%2Fiu C1949_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:63033__yw%2Fiu, base_url=http://127.0.0.1:63033/_yw/iu, leader=true}
[junit4:junit4]   2> 796643 T1814 C1949 P63033 oasc.SolrCore.execute [collection1] webapp=/_yw/iu path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 796645 T1814 C1949 P63033 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 796645 T1814 C1949 P63033 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4]   2> 796646 T1814 C1949 P63033 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 796647 T1814 C1949 P63033 oasup.LogUpdateProcessor.finish [collection1] webapp=/_yw/iu path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 2
[junit4:junit4]   2> 796648 T1894 C1948 P63074 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 796648 T1894 C1948 P63074 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 796650 T1815 C1949 P63033 oasc.SolrCore.execute [collection1] webapp=/_yw/iu path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 796650 T1894 C1948 P63074 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 796650 T1894 C1948 P63074 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 796651 T1894 C1948 P63074 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 796651 T1894 C1948 P63074 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 796655 T1894 C1948 P63074 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 797096 T1909 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4]   2> 797096 T1909 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4]   2> 797097 T1909 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 797106 T1909 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 797110 T1909 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 797129 T1909 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 797134 T1909 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 797146 T1909 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 797148 T1909 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 797148 T1909 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 797149 T1909 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 797150 T1909 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 797150 T1909 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 797150 T1909 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 797151 T1909 oasc.SolrCore.<init> [collection1] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty6-1372399548924\collection1\, dataDir=.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727/jetty6\
[junit4:junit4]   2> 797151 T1909 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@900bac2
[junit4:junit4]   2> 797151 T1909 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 797152 T1909 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727\jetty6
[junit4:junit4]   2> 797153 T1909 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727/jetty6\index/
[junit4:junit4]   2> 797154 T1909 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727\jetty6\index' doesn't exist. Creating new index...
[junit4:junit4]   2> 797154 T1909 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727\jetty6\index
[junit4:junit4]   2> 797166 T1909 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727\jetty6\index,segFN=segments_1,generation=1}
[junit4:junit4]   2> 797166 T1909 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 797172 T1909 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 797172 T1909 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 797174 T1909 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 797175 T1909 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 797176 T1909 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 797176 T1909 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 797180 T1909 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 797180 T1909 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 797180 T1909 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 797188 T1909 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 797190 T1804 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 797191 T1804 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"6",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63074__yw%2Fiu",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63074/_yw/iu"}
[junit4:junit4]   2> 797200 T1909 oass.SolrIndexSearcher.<init> Opening Searcher@161a520c main
[junit4:junit4]   2> 797201 T1908 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 797201 T1890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 797202 T1810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 797202 T1872 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 797203 T1803 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 797203 T1909 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 797203 T1855 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 797203 T1823 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 797203 T1839 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 797203 T1909 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 797213 T1909 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 797213 T1909 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:63087/_yw/iu collection:collection1 shard:shard2
[junit4:junit4]   2> 797214 T1911 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@161a520c main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 797219 T1909 oasc.ZkController.register We are http://127.0.0.1:63087/_yw/iu/collection1/ and leader is http://127.0.0.1:63042/_yw/iu/collection1/
[junit4:junit4]   2> 797219 T1909 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:63087/_yw/iu
[junit4:junit4]   2> 797220 T1909 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 797220 T1909 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C1950 name=collection1 org.apache.solr.core.SolrCore@755d4b47 url=http://127.0.0.1:63087/_yw/iu/collection1 node=127.0.0.1:63087__yw%2Fiu C1950_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:63087__yw%2Fiu, base_url=http://127.0.0.1:63087/_yw/iu}
[junit4:junit4]   2> 797220 T1912 C1950 P63087 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 797221 T1912 C1950 P63087 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 797221 T1912 C1950 P63087 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 797221 T1912 C1950 P63087 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 797221 T1909 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 797223 T1912 C1950 P63087 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 797224 T1781 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0
[junit4:junit4]   2> 797224 T1781 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 797224 T1781 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 797232 T1834 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 7, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4]   2> 797616 T1781 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 797619 T1781 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:63098
[junit4:junit4]   2> 797620 T1781 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 797621 T1781 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 797621 T1781 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty7-1372399553522
[junit4:junit4]   2> 797621 T1781 oasc.CoreContainer$Initializer.initialize looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty7-1372399553522\solr.xml
[junit4:junit4]   2> 797621 T1781 oasc.CoreContainer.<init> New CoreContainer 1865025355
[junit4:junit4]   2> 797621 T1781 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty7-1372399553522\'
[junit4:junit4]   2> 797622 T1781 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty7-1372399553522\'
[junit4:junit4]   2> 797715 T1781 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 797715 T1781 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 797716 T1781 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 797716 T1781 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 797717 T1781 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 797717 T1781 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 797718 T1781 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 797718 T1781 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 797718 T1781 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 797718 T1781 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 797724 T1781 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 797725 T1781 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:63014/solr
[junit4:junit4]   2> 797725 T1781 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 797727 T1781 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 797743 T1924 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@38929f9 name:ZooKeeperConnection Watcher:127.0.0.1:63014 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 797744 T1781 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 797747 T1781 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 797754 T1781 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 797758 T1926 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@705b9f19 name:ZooKeeperConnection Watcher:127.0.0.1:63014/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 797758 T1781 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 797765 T1781 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 798720 T1804 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 798722 T1804 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"7",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63087__yw%2Fiu",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63087/_yw/iu"}
[junit4:junit4]   2> 798740 T1803 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 798741 T1855 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 798742 T1823 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 798742 T1908 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 798743 T1890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 798744 T1810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 798743 T1839 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 798744 T1926 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 798744 T1872 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 798770 T1781 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63098__yw%2Fiu
[junit4:junit4]   2> 798773 T1781 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:63098__yw%2Fiu
[junit4:junit4]   2> 798777 T1855 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 798778 T1823 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 798778 T1908 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 798779 T1872 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 798779 T1926 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 798780 T1803 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4]   2> 798780 T1803 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 8)


[...truncated too long message...]

ite.lock FAILED !!!!!
[junit4:junit4]   2> !!!! WARNING: best effort to remove C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727\jetty2\index FAILED !!!!!
[junit4:junit4]   2> !!!! WARNING: best effort to remove C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727\jetty2 FAILED !!!!!
[junit4:junit4]   2> !!!! WARNING: best effort to remove C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1372399518727 FAILED !!!!!
[junit4:junit4]   2> 1166983 T1781 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4]   2> 1166985 T1781 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:63014 63014
[junit4:junit4]   2> 1168115 T1926 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 1169191 T1926 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   1> client port:0.0.0.0/0.0.0.0:0
[junit4:junit4]   1> safe stop:
[junit4:junit4]   2> 1169199 T1781 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> 1290861 T1781 oas.SolrTestCaseJ4.endTrackingSearchers ERROR ERROR: SolrIndexSearcher opens=35 closes=33
[junit4:junit4]   2> 528439 T1780 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4]   2> 548532 T1780 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 1 thread leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest: 
[junit4:junit4]   2> 	   1) Thread[id=1841, name=searcherExecutor-1322-thread-1, state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:957)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:917)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> 548533 T1780 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
[junit4:junit4]   2> 	   1) Thread[id=1841, name=searcherExecutor-1322-thread-1, state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
[junit4:junit4]   2> 551535 T1780 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
[junit4:junit4]   2> 	   1) Thread[id=1841, name=searcherExecutor-1322-thread-1, state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:957)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:917)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> NOTE: test params are: codec=FastCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST, chunkSize=241), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST, chunkSize=241)), sim=DefaultSimilarity, locale=en_NZ, timezone=America/Whitehorse
[junit4:junit4]   2> NOTE: Windows 7 6.1 amd64/Sun Microsystems Inc. 1.6.0_45 (64-bit)/cpus=2,threads=3,free=126520072,total=261386240
[junit4:junit4]   2> NOTE: All tests run in this JVM: [ChaosMonkeySafeLeaderTest, TestSchemaVersionResource, CollectionsAPIDistributedZkTest, SOLR749Test, ClusterStateTest, TestQuerySenderNoQuery, SolrInfoMBeanTest, TestFaceting, TestElisionMultitermQuery, AliasIntegrationTest, TimeZoneUtilsTest, BasicFunctionalityTest, TestDynamicFieldResource, TestIndexingPerformance, TestCollationKeyRangeQueries, ClusterStateUpdateTest, PrimitiveFieldTypeTest, OpenCloseCoreStressTest, TestRandomFaceting, TestSort, QueryResultKeyTest, TestClassNameShortening, IndexReaderFactoryTest, ResponseLogComponentTest, CircularListTest, TestLFUCache, PreAnalyzedFieldTest, TestOmitPositions, TestSolrDeletionPolicy1, TestUpdate, MBeansHandlerTest, HighlighterTest, TestArbitraryIndexDir, DirectUpdateHandlerOptimizeTest, CurrencyFieldOpenExchangeTest, OutputWriterTest, TestLMDirichletSimilarityFactory, PrimUtilsTest, TestZkChroot, OverseerCollectionProcessorTest, PluginInfoTest, TestLazyCores, TestComponentsName, DebugComponentTest, TestSolrQueryParserDefaultOperatorResource, XmlUpdateRequestHandlerTest, TestFoldingMultitermQuery, SuggesterTest, SearchHandlerTest, DocValuesMultiTest, PolyFieldTest, TermVectorComponentDistributedTest, TestRTGBase, FileUtilsTest, IndexBasedSpellCheckerTest, TestSolrXml, LoggingHandlerTest, TestPHPSerializedResponseWriter, ChaosMonkeyNothingIsSafeTest]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ChaosMonkeyNothingIsSafeTest -Dtests.seed=DC839F0952EE3A1E -Dtests.slow=true -Dtests.locale=en_NZ -Dtests.timezone=America/Whitehorse -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR   0.00s | ChaosMonkeyNothingIsSafeTest (suite) <<<
[junit4:junit4]    > Throwable #1: java.lang.AssertionError: ERROR: SolrIndexSearcher opens=35 closes=33
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([DC839F0952EE3A1E]:0)
[junit4:junit4]    > 	at org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:257)
[junit4:junit4]    > 	at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:104)
[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:662)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest: 
[junit4:junit4]    >    1) Thread[id=1841, name=searcherExecutor-1322-thread-1, state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
[junit4:junit4]    >         at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
[junit4:junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:957)
[junit4:junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:917)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([DC839F0952EE3A1E]:0)Throwable #3: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4:junit4]    >    1) Thread[id=1841, name=searcherExecutor-1322-thread-1, state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
[junit4:junit4]    >         at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
[junit4:junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:957)
[junit4:junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:917)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([DC839F0952EE3A1E]:0)
[junit4:junit4] Completed in 551.61s, 1 test, 1 failure, 2 errors <<< FAILURES!

[...truncated 723 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\build.xml:392: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\build.xml:372: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\build.xml:39: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build.xml:181: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\common-build.xml:437: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\lucene\common-build.xml:1248: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\lucene\common-build.xml:892: There were test failures: 299 suites, 1290 tests, 3 suite-level errors, 986 ignored (5 assumptions)

Total time: 43 minutes 40 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.6.0_45 -XX:-UseCompressedOops -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-4.x-Windows (64bit/jdk1.7.0_25) - Build # 2939 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Windows/2939/
Java: 64bit/jdk1.7.0_25 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.client.solrj.TestBatchUpdate.testWithBinary

Error Message:
IOException occured when talking to server at: https://127.0.0.1:55887/solr/collection1

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: https://127.0.0.1:55887/solr/collection1
	at __randomizedtesting.SeedInfo.seed([796FFE20C2E07FC5:2B19B03F450FA5D6]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:435)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
	at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.add(HttpSolrServer.java:572)
	at org.apache.solr.client.solrj.TestBatchUpdate.doIt(TestBatchUpdate.java:109)
	at org.apache.solr.client.solrj.TestBatchUpdate.testWithBinary(TestBatchUpdate.java:62)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:724)
Caused by: org.apache.http.NoHttpResponseException: The target server failed to respond
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:95)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:62)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:191)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:717)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:522)
	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
	... 45 more




Build Log:
[...truncated 10524 lines...]
[junit4:junit4] Suite: org.apache.solr.client.solrj.TestBatchUpdate
[junit4:junit4]   1> 23751 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.SolrJettyTestBase  ? Randomized ssl (true) and clientAuth (false)
[junit4:junit4]   1> 23752 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.SolrTestCaseJ4  ? ####initCore
[junit4:junit4]   1> 23754 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.SolrTestCaseJ4  ? ####initCore end
[junit4:junit4]   1> 23774 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.server.Server  ? jetty-8.1.10.v20130312
[junit4:junit4]   1> 24150 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.ssl.SslContextFactory  ? Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4:junit4]   1> 24161 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.server.AbstractConnector  ? Started SslSelectChannelConnector@127.0.0.1:55887
[junit4:junit4]   1> 24163 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.servlet.SolrDispatchFilter  ? SolrDispatchFilter.init()
[junit4:junit4]   1> 24166 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.core.SolrResourceLoader  ? JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   1> 24167 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.core.SolrResourceLoader  ? using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr
[junit4:junit4]   1> 24167 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.core.CoreContainer  ? looking for solr config file: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr\solr.xml
[junit4:junit4]   1> 24167 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.core.CoreContainer  ? New CoreContainer 1171430950
[junit4:junit4]   1> 24170 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.core.CoreContainer  ? Loading CoreContainer using Solr Home: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr\'
[junit4:junit4]   1> 24171 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.core.SolrResourceLoader  ? new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr\'
[junit4:junit4]   1> 24395 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.core.SolrCoreDiscoverer  ? Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr
[junit4:junit4]   1> 24398 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.core.SolrCoreDiscoverer  ? Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr\.svn
[junit4:junit4]   1> 24402 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.core.SolrCoreDiscoverer  ? Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr\.svn\prop-base
[junit4:junit4]   1> 24406 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.core.SolrCoreDiscoverer  ? Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr\.svn\props
[junit4:junit4]   1> 24408 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.core.SolrCoreDiscoverer  ? Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr\.svn\text-base
[junit4:junit4]   1> 24420 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.core.SolrCoreDiscoverer  ? Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr\.svn\tmp
[junit4:junit4]   1> 24422 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.core.SolrCoreDiscoverer  ? Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr\.svn\tmp\prop-base
[junit4:junit4]   1> 24424 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.core.SolrCoreDiscoverer  ? Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr\.svn\tmp\props
[junit4:junit4]   1> 24426 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.core.SolrCoreDiscoverer  ? Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr\.svn\tmp\text-base
[junit4:junit4]   1> 24428 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.core.SolrCoreDiscoverer  ? Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr\bin
[junit4:junit4]   1> 24430 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.core.SolrCoreDiscoverer  ? Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr\bin\.svn
[junit4:junit4]   1> 24434 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.core.SolrCoreDiscoverer  ? Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr\bin\.svn\prop-base
[junit4:junit4]   1> 24435 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.core.SolrCoreDiscoverer  ? Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr\bin\.svn\props
[junit4:junit4]   1> 24436 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.core.SolrCoreDiscoverer  ? Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr\bin\.svn\text-base
[junit4:junit4]   1> 24451 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.core.SolrCoreDiscoverer  ? Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr\bin\.svn\tmp
[junit4:junit4]   1> 24453 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.core.SolrCoreDiscoverer  ? Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr\bin\.svn\tmp\prop-base
[junit4:junit4]   1> 24457 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.core.SolrCoreDiscoverer  ? Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr\bin\.svn\tmp\props
[junit4:junit4]   1> 24460 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.core.SolrCoreDiscoverer  ? Looking for cores in C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr\bin\.svn\tmp\text-base
[junit4:junit4]   1> 24461 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.core.SolrCoreDiscoverer  ? Discovered properties file C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr\collection1\core.properties, adding to cores
[junit4:junit4]   1> 24469 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.handler.component.HttpShardHandlerFactory  ? Setting socketTimeout to: 0
[junit4:junit4]   1> 24469 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.handler.component.HttpShardHandlerFactory  ? Setting urlScheme to: http://
[junit4:junit4]   1> 24471 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.handler.component.HttpShardHandlerFactory  ? Setting connTimeout to: 0
[junit4:junit4]   1> 24471 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.handler.component.HttpShardHandlerFactory  ? Setting maxConnectionsPerHost to: 20
[junit4:junit4]   1> 24471 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.handler.component.HttpShardHandlerFactory  ? Setting corePoolSize to: 0
[junit4:junit4]   1> 24471 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.handler.component.HttpShardHandlerFactory  ? Setting maximumPoolSize to: 2147483647
[junit4:junit4]   1> 24472 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.handler.component.HttpShardHandlerFactory  ? Setting maxThreadIdleTime to: 5
[junit4:junit4]   1> 24473 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.handler.component.HttpShardHandlerFactory  ? Setting sizeOfQueue to: -1
[junit4:junit4]   1> 24475 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.handler.component.HttpShardHandlerFactory  ? Setting fairnessPolicy to: false
[junit4:junit4]   1> 24475 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.client.solrj.impl.HttpClientUtil  ? Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   1> 24485 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.logging.LogWatcher  ? Registering Log Listener
[junit4:junit4]   1> 24492 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.CoreContainer  ? Creating SolrCore 'collection1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr\collection1
[junit4:junit4]   1> 24493 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.SolrResourceLoader  ? new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr\collection1\'
[junit4:junit4]   1> 24594 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.SolrConfig  ? Adding specified lib dirs to ClassLoader
[junit4:junit4]   1> 24595 [coreLoadExecutor-33-thread-1] WARN  org.apache.solr.core.SolrResourceLoader  ? No files added to classloader from lib: ../../../contrib/extraction/lib (resolved as: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr\collection1\..\..\..\contrib\extraction\lib).
[junit4:junit4]   1> 24597 [coreLoadExecutor-33-thread-1] WARN  org.apache.solr.core.SolrResourceLoader  ? Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr\collection1\..\..\..\dist).
[junit4:junit4]   1> 24597 [coreLoadExecutor-33-thread-1] WARN  org.apache.solr.core.SolrResourceLoader  ? No files added to classloader from lib: ../../../contrib/clustering/lib/ (resolved as: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr\collection1\..\..\..\contrib\clustering\lib).
[junit4:junit4]   1> 24598 [coreLoadExecutor-33-thread-1] WARN  org.apache.solr.core.SolrResourceLoader  ? Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr\collection1\..\..\..\dist).
[junit4:junit4]   1> 24599 [coreLoadExecutor-33-thread-1] WARN  org.apache.solr.core.SolrResourceLoader  ? No files added to classloader from lib: ../../../contrib/langid/lib/ (resolved as: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr\collection1\..\..\..\contrib\langid\lib).
[junit4:junit4]   1> 24599 [coreLoadExecutor-33-thread-1] WARN  org.apache.solr.core.SolrResourceLoader  ? Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr\collection1\..\..\..\dist).
[junit4:junit4]   1> 24604 [coreLoadExecutor-33-thread-1] WARN  org.apache.solr.core.SolrResourceLoader  ? No files added to classloader from lib: ../../../contrib/velocity/lib (resolved as: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr\collection1\..\..\..\contrib\velocity\lib).
[junit4:junit4]   1> 24606 [coreLoadExecutor-33-thread-1] WARN  org.apache.solr.core.SolrResourceLoader  ? Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr\collection1\..\..\..\dist).
[junit4:junit4]   1> 24689 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.SolrConfig  ? Using Lucene MatchVersion: LUCENE_43
[junit4:junit4]   1> 24852 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.Config  ? Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   1> 24854 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.schema.IndexSchema  ? Reading Solr Schema from schema.xml
[junit4:junit4]   1> 24911 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.schema.IndexSchema  ? [collection1] Schema name=example
[junit4:junit4]   1> 26236 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.schema.IndexSchema  ? unique key field: id
[junit4:junit4]   1> 26270 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.schema.FileExchangeRateProvider  ? Reloading exchange rates from file currency.xml
[junit4:junit4]   1> 26283 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.schema.FileExchangeRateProvider  ? Reloading exchange rates from file currency.xml
[junit4:junit4]   1> 26371 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.SolrCore  ? org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   1> 26372 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.SolrCore  ? [collection1] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr\collection1\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-solrj\test\J0\solrtest-TestBatchUpdate-1372409765014\
[junit4:junit4]   1> 26372 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.JmxMonitoredMap  ? JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@422629f1
[junit4:junit4]   1> 26399 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.SolrCore  ? [collection1] Added SolrEventListener for newSearcher: org.apache.solr.core.QuerySenderListener{queries=[]}
[junit4:junit4]   1> 26399 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.SolrCore  ? [collection1] Added SolrEventListener for firstSearcher: org.apache.solr.core.QuerySenderListener{queries=[{q=static firstSearcher warming in solrconfig.xml}]}
[junit4:junit4]   1> 26400 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.SolrCore  ? org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   1> 26400 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.CachingDirectoryFactory  ? return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-solrj\test\J0\solrtest-TestBatchUpdate-1372409765014\
[junit4:junit4]   1> 26400 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.SolrCore  ? New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-solrj\test\J0\solrtest-TestBatchUpdate-1372409765014\index/
[junit4:junit4]   1> 26417 [coreLoadExecutor-33-thread-1] WARN  org.apache.solr.core.SolrCore  ? [collection1] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-solrj\test\J0\solrtest-TestBatchUpdate-1372409765014\index' doesn't exist. Creating new index...
[junit4:junit4]   1> 26420 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.CachingDirectoryFactory  ? return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-solrj\test\J0\solrtest-TestBatchUpdate-1372409765014\index
[junit4:junit4]   1> 26424 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.SolrCore  ? SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@76fbbfc6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@30b02343),segFN=segments_1,generation=1}
[junit4:junit4]   1> 26425 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.SolrCore  ? newest commit generation = 1
[junit4:junit4]   1> 26427 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.SolrCore  ? created json: solr.JSONResponseWriter
[junit4:junit4]   1> 26428 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.SolrCore  ? adding lazy queryResponseWriter: solr.VelocityResponseWriter
[junit4:junit4]   1> 26428 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.SolrCore  ? created velocity: solr.VelocityResponseWriter
[junit4:junit4]   1> 26428 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.SolrCore  ? created xslt: solr.XSLTResponseWriter
[junit4:junit4]   1> 26428 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.response.XSLTResponseWriter  ? xsltCacheLifetimeSeconds=5
[junit4:junit4]   1> 26435 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.RequestHandlers  ? created /select: solr.SearchHandler
[junit4:junit4]   1> 26436 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.RequestHandlers  ? created /query: solr.SearchHandler
[junit4:junit4]   1> 26437 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.RequestHandlers  ? created /get: solr.RealTimeGetHandler
[junit4:junit4]   1> 26439 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.RequestHandlers  ? created /browse: solr.SearchHandler
[junit4:junit4]   1> 26440 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.RequestHandlers  ? created /update: solr.UpdateRequestHandler
[junit4:junit4]   1> 26441 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.RequestHandlers  ? created /update/json: solr.JsonUpdateRequestHandler
[junit4:junit4]   1> 26443 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.RequestHandlers  ? created /update/csv: solr.CSVRequestHandler
[junit4:junit4]   1> 26443 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.RequestHandlers  ? adding lazy requestHandler: solr.extraction.ExtractingRequestHandler
[junit4:junit4]   1> 26444 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.RequestHandlers  ? created /update/extract: solr.extraction.ExtractingRequestHandler
[junit4:junit4]   1> 26444 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.RequestHandlers  ? adding lazy requestHandler: solr.FieldAnalysisRequestHandler
[junit4:junit4]   1> 26446 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.RequestHandlers  ? created /analysis/field: solr.FieldAnalysisRequestHandler
[junit4:junit4]   1> 26446 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.RequestHandlers  ? adding lazy requestHandler: solr.DocumentAnalysisRequestHandler
[junit4:junit4]   1> 26448 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.RequestHandlers  ? created /analysis/document: solr.DocumentAnalysisRequestHandler
[junit4:junit4]   1> 26448 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.RequestHandlers  ? created /admin/: solr.admin.AdminHandlers
[junit4:junit4]   1> 26449 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.RequestHandlers  ? created /admin/ping: solr.PingRequestHandler
[junit4:junit4]   1> 26450 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.RequestHandlers  ? created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   1> 26453 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.RequestHandlers  ? created /replication: solr.ReplicationHandler
[junit4:junit4]   1> 26453 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.RequestHandlers  ? adding lazy requestHandler: solr.SearchHandler
[junit4:junit4]   1> 26453 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.RequestHandlers  ? created /spell: solr.SearchHandler
[junit4:junit4]   1> 26454 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.RequestHandlers  ? adding lazy requestHandler: solr.SearchHandler
[junit4:junit4]   1> 26454 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.RequestHandlers  ? created /tvrh: solr.SearchHandler
[junit4:junit4]   1> 26456 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.RequestHandlers  ? adding lazy requestHandler: solr.SearchHandler
[junit4:junit4]   1> 26456 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.RequestHandlers  ? created /terms: solr.SearchHandler
[junit4:junit4]   1> 26456 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.RequestHandlers  ? adding lazy requestHandler: solr.SearchHandler
[junit4:junit4]   1> 26458 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.RequestHandlers  ? created /elevate: solr.SearchHandler
[junit4:junit4]   1> 26472 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.handler.loader.XMLLoader  ? xsltCacheLifetimeSeconds=60
[junit4:junit4]   1> 26474 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.handler.loader.XMLLoader  ? xsltCacheLifetimeSeconds=60
[junit4:junit4]   1> 26478 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.handler.loader.XMLLoader  ? xsltCacheLifetimeSeconds=60
[junit4:junit4]   1> 26746 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.search.SolrIndexSearcher  ? Opening Searcher@7e761ff0 main
[junit4:junit4]   1> 26747 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.update.UpdateHandler  ? Clearing tlog files, tlogDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-solrj\test\J0\solrtest-TestBatchUpdate-1372409765014\tlog
[junit4:junit4]   1> 26749 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.SolrCore  ? Hard AutoCommit: if uncommited for 15000ms; 
[junit4:junit4]   1> 26750 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.SolrCore  ? Soft AutoCommit: disabled
[junit4:junit4]   1> 26750 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.handler.component.SpellCheckComponent  ? Initializing spell checkers
[junit4:junit4]   1> 26753 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.spelling.DirectSolrSpellChecker  ? init: {name=default,field=text,classname=solr.DirectSolrSpellChecker,distanceMeasure=internal,accuracy=0.5,maxEdits=2,minPrefix=1,maxInspections=5,minQueryLength=4,maxQueryFrequency=0.01}
[junit4:junit4]   1> 26753 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.handler.component.SpellCheckComponent  ? No queryConverter defined, using default converter
[junit4:junit4]   1> 26754 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.handler.component.QueryElevationComponent  ? Loading QueryElevation from: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\example\solr\collection1\conf\elevate.xml
[junit4:junit4]   1> 26786 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.handler.ReplicationHandler  ? Commits will be reserved for  10000
[junit4:junit4]   1> 26799 [searcherExecutor-34-thread-1] INFO  org.apache.solr.core.SolrCore  ? QuerySenderListener sending requests to Searcher@7e761ff0 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   1> 26804 [searcherExecutor-34-thread-1] INFO  org.apache.solr.core.SolrCore  ? [collection1] webapp=null path=null params={event=firstSearcher&q=static+firstSearcher+warming+in+solrconfig.xml&distrib=false} hits=0 status=0 QTime=2 
[junit4:junit4]   1> 26804 [searcherExecutor-34-thread-1] INFO  org.apache.solr.core.SolrCore  ? QuerySenderListener done.
[junit4:junit4]   1> 26806 [searcherExecutor-34-thread-1] INFO  org.apache.solr.handler.component.SpellCheckComponent  ? Loading spell index for spellchecker: default
[junit4:junit4]   1> 26806 [searcherExecutor-34-thread-1] INFO  org.apache.solr.handler.component.SpellCheckComponent  ? Loading spell index for spellchecker: wordbreak
[junit4:junit4]   1> 26808 [searcherExecutor-34-thread-1] INFO  org.apache.solr.core.SolrCore  ? [collection1] Registered new searcher Searcher@7e761ff0 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   1> 26814 [coreLoadExecutor-33-thread-1] INFO  org.apache.solr.core.CoreContainer  ? registering core: collection1
[junit4:junit4]   1> 26815 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.servlet.SolrDispatchFilter  ? user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-solrj\test\J0
[junit4:junit4]   1> 26816 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.servlet.SolrDispatchFilter  ? SolrDispatchFilter.init() done
[junit4:junit4]   1> 26816 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.SolrJettyTestBase  ? Jetty Assigned Port#55887
[junit4:junit4]   2> Creating dataDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-solrj\test\J0\.\solrtest-TestBatchUpdate-1372409765014
[junit4:junit4]   1> 27034 [TEST-TestBatchUpdate.testWithXml-seed#[796FFE20C2E07FC5]] INFO  org.apache.solr.SolrTestCaseJ4  ? ###Starting testWithXml
[junit4:junit4]   1> 27038 [TEST-TestBatchUpdate.testWithXml-seed#[796FFE20C2E07FC5]] INFO  org.apache.solr.client.solrj.impl.HttpClientUtil  ? Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   1> 27317 [qtp979168610-73] INFO  org.apache.solr.core.SolrCore  ? SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@76fbbfc6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@30b02343),segFN=segments_1,generation=1}
[junit4:junit4]   1> 27320 [qtp979168610-73] INFO  org.apache.solr.core.SolrCore  ? newest commit generation = 1
[junit4:junit4]   1> 27324 [qtp979168610-73] INFO  org.apache.solr.search.SolrIndexSearcher  ? Opening Searcher@550b37bf realtime
[junit4:junit4]   1> 27326 [qtp979168610-73] INFO  org.apache.solr.update.processor.LogUpdateProcessor  ? [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1439075945492250624)} 0 14
[junit4:junit4]   1> 28459 [qtp979168610-74] INFO  org.apache.solr.update.processor.LogUpdateProcessor  ? [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1 (1439075945578233856), 2 (1439075945582428160), 3 (1439075945583476736), 4 (1439075945586622464), 5 (1439075945586622465), 6 (1439075945587671040), 7 (1439075945615982592), 8 (1439075945617031168), 9 (1439075945618079744), 10 (1439075945618079745), ... (1000 adds)]} 0 1065
[junit4:junit4]   1> 28469 [qtp979168610-72] INFO  org.apache.solr.update.UpdateHandler  ? start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   1> 28569 [qtp979168610-72] INFO  org.apache.solr.core.SolrCore  ? SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@76fbbfc6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@30b02343),segFN=segments_1,generation=1}
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@76fbbfc6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@30b02343),segFN=segments_2,generation=2}
[junit4:junit4]   1> 28571 [qtp979168610-72] INFO  org.apache.solr.core.SolrCore  ? newest commit generation = 2
[junit4:junit4]   1> 28575 [qtp979168610-72] INFO  org.apache.solr.search.SolrIndexSearcher  ? Opening Searcher@3726fdb main
[junit4:junit4]   1> 28575 [qtp979168610-72] INFO  org.apache.solr.update.UpdateHandler  ? end_commit_flush
[junit4:junit4]   1> 28576 [searcherExecutor-34-thread-1] INFO  org.apache.solr.core.SolrCore  ? QuerySenderListener sending requests to Searcher@3726fdb main{StandardDirectoryReader(segments_2:3:nrt _0(4.4):C1000)}
[junit4:junit4]   1> 28576 [searcherExecutor-34-thread-1] INFO  org.apache.solr.core.SolrCore  ? QuerySenderListener done.
[junit4:junit4]   1> 28581 [searcherExecutor-34-thread-1] INFO  org.apache.solr.core.SolrCore  ? [collection1] Registered new searcher Searcher@3726fdb main{StandardDirectoryReader(segments_2:3:nrt _0(4.4):C1000)}
[junit4:junit4]   1> 28584 [qtp979168610-72] INFO  org.apache.solr.update.processor.LogUpdateProcessor  ? [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 115
[junit4:junit4]   1> 28608 [qtp979168610-73] INFO  org.apache.solr.core.SolrCore  ? [collection1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=1000 status=0 QTime=3 
[junit4:junit4]   1> 28617 [TEST-TestBatchUpdate.testWithXml-seed#[796FFE20C2E07FC5]] INFO  org.apache.solr.SolrTestCaseJ4  ? ###Ending testWithXml
[junit4:junit4]   1> 28653 [TEST-TestBatchUpdate.testWithBinaryBean-seed#[796FFE20C2E07FC5]] INFO  org.apache.solr.SolrTestCaseJ4  ? ###Starting testWithBinaryBean
[junit4:junit4]   1> 28810 [qtp979168610-75] INFO  org.apache.solr.search.SolrIndexSearcher  ? Opening Searcher@3b0af467 realtime
[junit4:junit4]   1> 28812 [qtp979168610-75] INFO  org.apache.solr.update.processor.LogUpdateProcessor  ? [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1439075946920411136)} 0 144
[junit4:junit4]   1> 29284 [qtp979168610-76] INFO  org.apache.solr.update.processor.LogUpdateProcessor  ? [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1 (1439075947106009088), 2 (1439075947109154816), 3 (1439075947109154817), 4 (1439075947109154818), 5 (1439075947110203392), 6 (1439075947110203393), 7 (1439075947110203394), 8 (1439075947111251968), 9 (1439075947111251969), 10 (1439075947111251970), ... (1000 adds)]} 0 433
[junit4:junit4]   1> 29290 [qtp979168610-71] INFO  org.apache.solr.update.UpdateHandler  ? start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   1> 29635 [qtp979168610-71] INFO  org.apache.solr.core.SolrCore  ? SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@76fbbfc6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@30b02343),segFN=segments_2,generation=2}
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@76fbbfc6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@30b02343),segFN=segments_3,generation=3}
[junit4:junit4]   1> 29635 [qtp979168610-71] INFO  org.apache.solr.core.SolrCore  ? newest commit generation = 3
[junit4:junit4]   1> 29639 [qtp979168610-71] INFO  org.apache.solr.search.SolrIndexSearcher  ? Opening Searcher@e705c11 main
[junit4:junit4]   1> 29640 [qtp979168610-71] INFO  org.apache.solr.update.UpdateHandler  ? end_commit_flush
[junit4:junit4]   1> 29642 [searcherExecutor-34-thread-1] INFO  org.apache.solr.core.SolrCore  ? QuerySenderListener sending requests to Searcher@e705c11 main{StandardDirectoryReader(segments_3:7:nrt _1(4.4):C1000)}
[junit4:junit4]   1> 29642 [searcherExecutor-34-thread-1] INFO  org.apache.solr.core.SolrCore  ? QuerySenderListener done.
[junit4:junit4]   1> 29647 [searcherExecutor-34-thread-1] INFO  org.apache.solr.core.SolrCore  ? [collection1] Registered new searcher Searcher@e705c11 main{StandardDirectoryReader(segments_3:7:nrt _1(4.4):C1000)}
[junit4:junit4]   1> 29651 [qtp979168610-71] INFO  org.apache.solr.update.processor.LogUpdateProcessor  ? [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 361
[junit4:junit4]   1> 29660 [qtp979168610-72] INFO  org.apache.solr.core.SolrCore  ? [collection1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=1000 status=0 QTime=1 
[junit4:junit4]   1> 29665 [TEST-TestBatchUpdate.testWithBinaryBean-seed#[796FFE20C2E07FC5]] INFO  org.apache.solr.SolrTestCaseJ4  ? ###Ending testWithBinaryBean
[junit4:junit4]   1> 29691 [TEST-TestBatchUpdate.testWithBinary-seed#[796FFE20C2E07FC5]] INFO  org.apache.solr.SolrTestCaseJ4  ? ###Starting testWithBinary
[junit4:junit4]   1> 29721 [qtp979168610-75] INFO  org.apache.solr.search.SolrIndexSearcher  ? Opening Searcher@1b1789b1 realtime
[junit4:junit4]   1> 29722 [qtp979168610-75] INFO  org.apache.solr.update.processor.LogUpdateProcessor  ? [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1439075947993104384)} 0 25
[junit4:junit4]   1> 47134 [commitScheduler-36-thread-1] INFO  org.apache.solr.update.UpdateHandler  ? start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   1> 47135 [commitScheduler-36-thread-1] INFO  org.apache.solr.core.SolrCore  ? SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@76fbbfc6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@30b02343),segFN=segments_3,generation=3}
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@76fbbfc6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@30b02343),segFN=segments_4,generation=4}
[junit4:junit4]   1> 47137 [commitScheduler-36-thread-1] INFO  org.apache.solr.core.SolrCore  ? newest commit generation = 4
[junit4:junit4]   1> 47138 [commitScheduler-36-thread-1] INFO  org.apache.solr.update.UpdateHandler  ? end_commit_flush
[junit4:junit4]   1> 229928 [TEST-TestBatchUpdate.testWithBinary-seed#[796FFE20C2E07FC5]] INFO  org.apache.solr.SolrTestCaseJ4  ? ###Ending testWithBinary
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestBatchUpdate -Dtests.method=testWithBinary -Dtests.seed=796FFE20C2E07FC5 -Dtests.slow=true -Dtests.locale=zh_SG -Dtests.timezone=EAT -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR    200s | TestBatchUpdate.testWithBinary <<<
[junit4:junit4]    > Throwable #1: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: https://127.0.0.1:55887/solr/collection1
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([796FFE20C2E07FC5:2B19B03F450FA5D6]:0)
[junit4:junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:435)
[junit4:junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4]    > 	at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
[junit4:junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.add(HttpSolrServer.java:572)
[junit4:junit4]    > 	at org.apache.solr.client.solrj.TestBatchUpdate.doIt(TestBatchUpdate.java:109)
[junit4:junit4]    > 	at org.apache.solr.client.solrj.TestBatchUpdate.testWithBinary(TestBatchUpdate.java:62)
[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:724)
[junit4:junit4]    > Caused by: org.apache.http.NoHttpResponseException: The target server failed to respond
[junit4:junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:95)
[junit4:junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:62)
[junit4:junit4]    > 	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254)
[junit4:junit4]    > 	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289)
[junit4:junit4]    > 	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252)
[junit4:junit4]    > 	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:191)
[junit4:junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300)
[junit4:junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127)
[junit4:junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:717)
[junit4:junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:522)
[junit4:junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4]    > 	... 45 more
[junit4:junit4]   1> 230173 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.core.CoreContainer  ? Shutting down CoreContainer instance=1171430950
[junit4:junit4]   1> 230176 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.server.handler.ContextHandler  ? stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4]   1> 230395 [qtp979168610-76] ERROR org.apache.solr.core.SolrCore  ? org.apache.solr.common.SolrException: java.nio.channels.ClosedByInterruptException
[junit4:junit4]   1> 	at org.apache.solr.update.TransactionLog.finish(TransactionLog.java:527)
[junit4:junit4]   1> 	at org.apache.solr.update.UpdateLog.finish(UpdateLog.java:754)
[junit4:junit4]   1> 	at org.apache.solr.update.processor.RunUpdateProcessor.finish(RunUpdateProcessorFactory.java:115)
[junit4:junit4]   1> 	at org.apache.solr.update.processor.DistributedUpdateProcessor.finish(DistributedUpdateProcessor.java:1187)
[junit4:junit4]   1> 	at org.apache.solr.update.processor.LogUpdateProcessor.finish(LogUpdateProcessorFactory.java:179)
[junit4:junit4]   1> 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:83)
[junit4:junit4]   1> 	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
[junit4:junit4]   1> 	at org.apache.solr.core.SolrCore.execute(SolrCore.java:1894)
[junit4:junit4]   1> 	at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:656)
[junit4:junit4]   1> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:359)
[junit4:junit4]   1> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:155)
[junit4:junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
[junit4:junit4]   1> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:135)
[junit4:junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
[junit4:junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077)
[junit4:junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.Server.handle(Server.java:368)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)
[junit4:junit4]   1> 	at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:953)
[junit4:junit4]   1> 	at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
[junit4:junit4]   1> 	at org.eclipse.jetty.io.nio.SslConnection.handle(SslConnection.java:196)
[junit4:junit4]   1> 	at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
[junit4:junit4]   1> 	at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
[junit4:junit4]   1> 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   1> 	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   1> 	at java.lang.Thread.run(Thread.java:724)
[junit4:junit4]   1> Caused by: java.nio.channels.ClosedByInterruptException
[junit4:junit4]   1> 	at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
[junit4:junit4]   1> 	at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:200)
[junit4:junit4]   1> 	at java.nio.channels.Channels.writeFullyImpl(Channels.java:78)
[junit4:junit4]   1> 	at java.nio.channels.Channels.writeFully(Channels.java:101)
[junit4:junit4]   1> 	at java.nio.channels.Channels.access$000(Channels.java:61)
[junit4:junit4]   1> 	at java.nio.channels.Channels$1.write(Channels.java:174)
[junit4:junit4]   1> 	at org.apache.solr.common.util.FastOutputStream.flush(FastOutputStream.java:214)
[junit4:junit4]   1> 	at org.apache.solr.common.util.FastOutputStream.flushBuffer(FastOutputStream.java:207)
[junit4:junit4]   1> 	at org.apache.solr.update.TransactionLog.finish(TransactionLog.java:516)
[junit4:junit4]   1> 	... 36 more
[junit4:junit4]   1> 
[junit4:junit4]   1> 230397 [qtp979168610-76] INFO  org.apache.solr.core.SolrCore  ? [collection1] webapp=/solr path=/update params={wt=javabin&version=2} status=500 QTime=473 
[junit4:junit4]   1> 230399 [qtp979168610-76] ERROR org.apache.solr.servlet.SolrDispatchFilter  ? null:org.apache.solr.common.SolrException: java.nio.channels.ClosedByInterruptException
[junit4:junit4]   1> 	at org.apache.solr.update.TransactionLog.finish(TransactionLog.java:527)
[junit4:junit4]   1> 	at org.apache.solr.update.UpdateLog.finish(UpdateLog.java:754)
[junit4:junit4]   1> 	at org.apache.solr.update.processor.RunUpdateProcessor.finish(RunUpdateProcessorFactory.java:115)
[junit4:junit4]   1> 	at org.apache.solr.update.processor.DistributedUpdateProcessor.finish(DistributedUpdateProcessor.java:1187)
[junit4:junit4]   1> 	at org.apache.solr.update.processor.LogUpdateProcessor.finish(LogUpdateProcessorFactory.java:179)
[junit4:junit4]   1> 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:83)
[junit4:junit4]   1> 	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
[junit4:junit4]   1> 	at org.apache.solr.core.SolrCore.execute(SolrCore.java:1894)
[junit4:junit4]   1> 	at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:656)
[junit4:junit4]   1> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:359)
[junit4:junit4]   1> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:155)
[junit4:junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
[junit4:junit4]   1> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:135)
[junit4:junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
[junit4:junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077)
[junit4:junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.Server.handle(Server.java:368)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)
[junit4:junit4]   1> 	at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:953)
[junit4:junit4]   1> 	at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
[junit4:junit4]   1> 	at org.eclipse.jetty.io.nio.SslConnection.handle(SslConnection.java:196)
[junit4:junit4]   1> 	at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
[junit4:junit4]   1> 	at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
[junit4:junit4]   1> 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   1> 	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   1> 	at java.lang.Thread.run(Thread.java:724)
[junit4:junit4]   1> Caused by: java.nio.channels.ClosedByInterruptException
[junit4:junit4]   1> 	at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
[junit4:junit4]   1> 	at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:200)
[junit4:junit4]   1> 	at java.nio.channels.Channels.writeFullyImpl(Channels.java:78)
[junit4:junit4]   1> 	at java.nio.channels.Channels.writeFully(Channels.java:101)
[junit4:junit4]   1> 	at java.nio.channels.Channels.access$000(Channels.java:61)
[junit4:junit4]   1> 	at java.nio.channels.Channels$1.write(Channels.java:174)
[junit4:junit4]   1> 	at org.apache.solr.common.util.FastOutputStream.flush(FastOutputStream.java:214)
[junit4:junit4]   1> 	at org.apache.solr.common.util.FastOutputStream.flushBuffer(FastOutputStream.java:207)
[junit4:junit4]   1> 	at org.apache.solr.update.TransactionLog.finish(TransactionLog.java:516)
[junit4:junit4]   1> 	... 36 more
[junit4:junit4]   1> 
[junit4:junit4]   1> 230402 [qtp979168610-76] ERROR org.apache.solr.servlet.SolrDispatchFilter  ? null:org.eclipse.jetty.io.EofException
[junit4:junit4]   1> 	at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:142)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:107)
[junit4:junit4]   1> 	at org.apache.solr.common.util.FastOutputStream.flush(FastOutputStream.java:214)
[junit4:junit4]   1> 	at org.apache.solr.common.util.FastOutputStream.flushBuffer(FastOutputStream.java:207)
[junit4:junit4]   1> 	at org.apache.solr.common.util.JavaBinCodec.marshal(JavaBinCodec.java:94)
[junit4:junit4]   1> 	at org.apache.solr.response.BinaryResponseWriter.write(BinaryResponseWriter.java:50)
[junit4:junit4]   1> 	at org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:637)
[junit4:junit4]   1> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:372)
[junit4:junit4]   1> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:155)
[junit4:junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
[junit4:junit4]   1> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:135)
[junit4:junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
[junit4:junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077)
[junit4:junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.Server.handle(Server.java:368)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)
[junit4:junit4]   1> 	at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:953)
[junit4:junit4]   1> 	at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
[junit4:junit4]   1> 	at org.eclipse.jetty.io.nio.SslConnection.handle(SslConnection.java:196)
[junit4:junit4]   1> 	at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
[junit4:junit4]   1> 	at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
[junit4:junit4]   1> 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   1> 	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   1> 	at java.lang.Thread.run(Thread.java:724)
[junit4:junit4]   1> 
[junit4:junit4]   1> 230422 [qtp979168610-76] ERROR org.apache.solr.servlet.SolrDispatchFilter  ? null:org.eclipse.jetty.io.EofException
[junit4:junit4]   1> 	at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:142)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:107)
[junit4:junit4]   1> 	at org.apache.solr.common.util.FastOutputStream.flush(FastOutputStream.java:214)
[junit4:junit4]   1> 	at org.apache.solr.common.util.FastOutputStream.flushBuffer(FastOutputStream.java:207)
[junit4:junit4]   1> 	at org.apache.solr.common.util.JavaBinCodec.marshal(JavaBinCodec.java:94)
[junit4:junit4]   1> 	at org.apache.solr.response.BinaryResponseWriter.write(BinaryResponseWriter.java:50)
[junit4:junit4]   1> 	at org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:637)
[junit4:junit4]   1> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:372)
[junit4:junit4]   1> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:155)
[junit4:junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
[junit4:junit4]   1> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:135)
[junit4:junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
[junit4:junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077)
[junit4:junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.Server.handle(Server.java:368)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)
[junit4:junit4]   1> 	at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:953)
[junit4:junit4]   1> 	at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
[junit4:junit4]   1> 	at org.eclipse.jetty.io.nio.SslConnection.handle(SslConnection.java:196)
[junit4:junit4]   1> 	at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
[junit4:junit4]   1> 	at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
[junit4:junit4]   1> 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   1> 	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   1> 	at java.lang.Thread.run(Thread.java:724)
[junit4:junit4]   1> 
[junit4:junit4]   1> 230424 [qtp979168610-76] INFO  org.apache.solr.core.SolrCore  ? [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@32862edb
[junit4:junit4]   1> 230396 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] WARN  org.eclipse.jetty.util.thread.QueuedThreadPool  ? 1 threads could not be stopped
[junit4:junit4]   1> 230436 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ? Couldn't stop Thread[qtp979168610-76,5,TGRP-TestBatchUpdate]
[junit4:junit4]   1> 230445 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at java.lang.Object.getClass(Native Method)
[junit4:junit4]   1> 230446 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at java.util.Arrays.copyOf(Arrays.java:2219)
[junit4:junit4]   1> 230446 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at java.util.ArrayList.grow(ArrayList.java:213)
[junit4:junit4]   1> 230447 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at java.util.ArrayList.ensureCapacityInternal(ArrayList.java:187)
[junit4:junit4]   1> 230447 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at java.util.ArrayList.add(ArrayList.java:411)
[junit4:junit4]   1> 230448 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at org.apache.solr.common.util.NamedList.add(NamedList.java:144)
[junit4:junit4]   1> 230448 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at org.apache.solr.handler.RequestHandlerBase.getStatistics(RequestHandlerBase.java:206)
[junit4:junit4]   1> 230448 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at org.apache.solr.core.JmxMonitoredMap$SolrDynamicMBean.getMBeanInfo(JmxMonitoredMap.java:231)
[junit4:junit4]   1> 230449 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getClassName(DefaultMBeanServerInterceptor.java:1804)
[junit4:junit4]   1> 230449 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
[junit4:junit4]   1> 230449 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
[junit4:junit4]   1> 230451 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
[junit4:junit4]   1> 230451 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
[junit4:junit4]   1> 230451 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
[junit4:junit4]   1> 230452 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at org.apache.solr.core.JmxMonitoredMap.unregister(JmxMonitoredMap.java:175)
[junit4:junit4]   1> 230452 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at org.apache.solr.core.JmxMonitoredMap.clear(JmxMonitoredMap.java:116)
[junit4:junit4]   1> 230453 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at org.apache.solr.core.SolrCore.close(SolrCore.java:1042)
[junit4:junit4]   1> 230453 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:389)
[junit4:junit4]   1> 230453 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:155)
[junit4:junit4]   1> 230454 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
[junit4:junit4]   1> 230454 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:135)
[junit4:junit4]   1> 230455 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
[junit4:junit4]   1> 230455 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
[junit4:junit4]   1> 230456 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
[junit4:junit4]   1> 230456 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
[junit4:junit4]   1> 230456 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301)
[junit4:junit4]   1> 230457 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077)
[junit4:junit4]   1> 230457 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
[junit4:junit4]   1> 230459 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
[junit4:junit4]   1> 230460 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
[junit4:junit4]   1> 230460 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
[junit4:junit4]   1> 230460 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
[junit4:junit4]   1> 230461 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at org.eclipse.jetty.server.Server.handle(Server.java:368)
[junit4:junit4]   1> 230462 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
[junit4:junit4]   1> 230462 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
[junit4:junit4]   1> 230462 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)
[junit4:junit4]   1> 230463 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:953)
[junit4:junit4]   1> 230464 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
[junit4:junit4]   1> 230465 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
[junit4:junit4]   1> 230475 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at org.eclipse.jetty.io.nio.SslConnection.handle(SslConnection.java:196)
[junit4:junit4]   1> 230476 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
[junit4:junit4]   1> 230476 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
[junit4:junit4]   1> 230476 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   1> 230477 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   1> 230478 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.eclipse.jetty.util.thread.QueuedThreadPool  ?  at java.lang.Thread.run(Thread.java:724)
[junit4:junit4]   1> 230479 [SUITE-TestBatchUpdate-seed#[796FFE20C2E07FC5]-worker] INFO  org.apache.solr.SolrTestCaseJ4  ? ###deleteCore
[junit4:junit4]   1> 230519 [qtp979168610-76] INFO  org.apache.solr.update.UpdateHandler  ? closing DirectUpdateHandler2{commits=3,autocommit maxTime=15000ms,autocommits=1,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=1000,adds=1000,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=3000,cumulative_deletesById=0,cumulative_deletesByQuery=3,cumulative_errors=0}
[junit4:junit4]   1> 230520 [qtp979168610-76] INFO  org.apache.solr.update.SolrCoreState  ? Closing SolrCoreState
[junit4:junit4]   1> 230521 [qtp979168610-76] INFO  org.apache.solr.update.DefaultSolrCoreState  ? SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   1> 230521 [qtp979168610-76] INFO  org.apache.solr.update.DefaultSolrCoreState  ? closing IndexWriter with IndexWriterCloser
[junit4:junit4]   1> 230585 [qtp979168610-76] INFO  org.apache.solr.core.SolrCore  ? SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@76fbbfc6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@30b02343),segFN=segments_4,generation=4}
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@76fbbfc6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@30b02343),segFN=segments_5,generation=5}
[junit4:junit4]   1> 230586 [qtp979168610-76] INFO  org.apache.solr.core.SolrCore  ? newest commit generation = 5
[junit4:junit4]   1> 230592 [qtp979168610-76] ERROR org.apache.solr.update.UpdateHandler  ? Error in final commit
[junit4:junit4]   1> org.apache.solr.common.SolrException: java.nio.channels.ClosedChannelException
[junit4:junit4]   1> 	at org.apache.solr.update.TransactionLog.writeCommit(TransactionLog.java:455)
[junit4:junit4]   1> 	at org.apache.solr.update.UpdateLog.postCommit(UpdateLog.java:605)
[junit4:junit4]   1> 	at org.apache.solr.update.DirectUpdateHandler2.closeWriter(DirectUpdateHandler2.java:739)
[junit4:junit4]   1> 	at org.apache.solr.update.DefaultSolrCoreState.closeIndexWriter(DefaultSolrCoreState.java:68)
[junit4:junit4]   1> 	at org.apache.solr.update.DefaultSolrCoreState.close(DefaultSolrCoreState.java:359)
[junit4:junit4]   1> 	at org.apache.solr.update.SolrCoreState.decrefSolrCoreState(SolrCoreState.java:72)
[junit4:junit4]   1> 	at org.apache.solr.core.SolrCore.close(SolrCore.java:1059)
[junit4:junit4]   1> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:389)
[junit4:junit4]   1> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:155)
[junit4:junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
[junit4:junit4]   1> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:135)
[junit4:junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
[junit4:junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077)
[junit4:junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.Server.handle(Server.java:368)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)
[junit4:junit4]   1> 	at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:953)
[junit4:junit4]   1> 	at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
[junit4:junit4]   1> 	at org.eclipse.jetty.io.nio.SslConnection.handle(SslConnection.java:196)
[junit4:junit4]   1> 	at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
[junit4:junit4]   1> 	at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
[junit4:junit4]   1> 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   1> 	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   1> 	at java.lang.Thread.run(Thread.java:724)
[junit4:junit4]   1> Caused by: java.nio.channels.ClosedChannelException
[junit4:junit4]   1> 	at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:94)
[junit4:junit4]   1> 	at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:183)
[junit4:junit4]   1> 	at java.nio.channels.Channels.writeFullyImpl(Channels.java:78)
[junit4:junit4]   1> 	at java.nio.channels.Channels.writeFully(Channels.java:101)
[junit4:junit4]   1> 	at java.nio.channels.Channels.access$000(Channels.java:61)
[junit4:junit4]   1> 	at java.nio.channels.Channels$1.write(Channels.java:174)
[junit4:junit4]   1> 	at org.apache.solr.common.util.FastOutputStream.flush(FastOutputStream.java:214)
[junit4:junit4]   1> 	at org.apache.solr.common.util.FastOutputStream.flushBuffer(FastOutputStream.java:207)
[junit4:junit4]   1> 	at org.apache.solr.common.util.FastOutputStream.flush(FastOutputStream.java:191)
[junit4:junit4]   1> 	at org.apache.solr.update.TransactionLog.writeCommit(TransactionLog.java:450)
[junit4:junit4]   1> 	... 34 more
[junit4:junit4]   1> 230605 [qtp979168610-76] ERROR org.apache.solr.update.UpdateHandler  ? Error closing log files
[junit4:junit4]   1> org.apache.solr.common.SolrException: java.nio.channels.ClosedChannelException
[junit4:junit4]   1> 	at org.apache.solr.update.TransactionLog.close(TransactionLog.java:546)
[junit4:junit4]   1> 	at org.apache.solr.update.TransactionLog.decref(TransactionLog.java:501)
[junit4:junit4]   1> 	at org.apache.solr.update.UpdateLog.doClose(UpdateLog.java:818)
[junit4:junit4]   1> 	at org.apache.solr.update.UpdateLog.close(UpdateLog.java:837)
[junit4:junit4]   1> 	at org.apache.solr.update.UpdateLog.close(UpdateLog.java:824)
[junit4:junit4]   1> 	at org.apache.solr.update.DirectUpdateHandler2.closeWriter(DirectUpdateHandler2.java:749)
[junit4:junit4]   1> 	at org.apache.solr.update.DefaultSolrCoreState.closeIndexWriter(DefaultSolrCoreState.java:68)
[junit4:junit4]   1> 	at org.apache.solr.update.DefaultSolrCoreState.close(DefaultSolrCoreState.java:359)
[junit4:junit4]   1> 	at org.apache.solr.update.SolrCoreState.decrefSolrCoreState(SolrCoreState.java:72)
[junit4:junit4]   1> 	at org.apache.solr.core.SolrCore.close(SolrCore.java:1059)
[junit4:junit4]   1> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:389)
[junit4:junit4]   1> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:155)
[junit4:junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
[junit4:junit4]   1> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:135)
[junit4:junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
[junit4:junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077)
[junit4:junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.Server.handle(Server.java:368)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)
[junit4:junit4]   1> 	at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:953)
[junit4:junit4]   1> 	at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
[junit4:junit4]   1> 	at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
[junit4:junit4]   1> 	at org.eclipse.jetty.io.nio.SslConnection.handle(SslConnection.java:196)
[junit4:junit4]   1> 	at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
[junit4:junit4]   1> 	at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
[junit4:junit4]   1> 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   1> 	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   1> 	at java.lang.Thread.run(Thread.java:724)
[junit4:junit4]   1> Caused by: java.nio.channels.ClosedChannelException
[junit4:junit4]   1> 	at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:94)
[junit4:junit4]   1> 	at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:183)
[junit4:junit4]   1> 	at java.nio.channels.Channels.writeFullyImpl(Channels.java:78)
[junit4:junit4]   1> 	at java.nio.channels.Channels.writeFully(Channels.java:101)
[junit4:junit4]   1> 	at java.nio.channels.Channels.access$000(Channels.java:61)
[junit4:junit4]   1> 	at java.nio.channels.Channels$1.write(Channels.java:174)
[junit4:junit4]   1> 	at org.apache.solr.common.util.FastOutputStream.flush(FastOutputStream.java:214)
[junit4:junit4]   1> 	at org.apache.solr.common.util.FastOutputStream.flushBuffer(FastOutputStream.java:207)
[junit4:junit4]   1> 	at org.apache.solr.common.util.FastOutputStream.flush(FastOutputStream.java:191)
[junit4:junit4]   1> 	at org.apache.solr.update.TransactionLog.close(TransactionLog.java:538)
[junit4:junit4]   1> 	... 37 more
[junit4:junit4]   1> 230660 [qtp979168610-76] INFO  org.apache.solr.core.SolrCore  ? [collection1] Closing main searcher on request.
[junit4:junit4]   1> 230662 [qtp979168610-76] INFO  org.apache.solr.core.CachingDirectoryFactory  ? Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4]   1> 230662 [qtp979168610-76] INFO  org.apache.solr.core.CachingDirectoryFactory  ? looking to close C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-solrj\test\J0\solrtest-TestBatchUpdate-1372409765014\index [CachedDir<<refCount=0;path=C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-solrj\test\J0\solrtest-TestBatchUpdate-1372409765014\index;done=false>>]
[junit4:junit4]   1> 230663 [qtp979168610-76] INFO  org.apache.solr.core.CachingDirectoryFactory  ? Closing directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-solrj\test\J0\solrtest-TestBatchUpdate-1372409765014\index
[junit4:junit4]   1> 230663 [qtp979168610-76] INFO  org.apache.solr.core.CachingDirectoryFactory  ? looking to close C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-solrj\test\J0\solrtest-TestBatchUpdate-1372409765014\ [CachedDir<<refCount=0;path=C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-solrj\test\J0\solrtest-TestBatchUpdate-1372409765014\;done=false>>]
[junit4:junit4]   1> 230665 [qtp979168610-76] INFO  org.apache.solr.core.CachingDirectoryFactory  ? Closing directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-solrj\test\J0\solrtest-TestBatchUpdate-1372409765014\
[junit4:junit4]   2> !!!! WARNING: best effort to remove C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-solrj\test\J0\.\solrtest-TestBatchUpdate-1372409765014\index\org.apache.lucene.store.RAMDirectory@76fbbfc6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@30b02343-write.lock FAILED !!!!!
[junit4:junit4]   2> !!!! WARNING: best effort to remove C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-solrj\test\J0\.\solrtest-TestBatchUpdate-1372409765014\index FAILED !!!!!
[junit4:junit4]   2> !!!! WARNING: best effort to remove C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-solrj\test\J0\.\solrtest-TestBatchUpdate-1372409765014 FAILED !!!!!
[junit4:junit4]   2> NOTE: test params are: codec=Lucene3x, sim=DefaultSimilarity, locale=zh_SG, timezone=EAT
[junit4:junit4]   2> NOTE: Windows 7 6.1 amd64/Oracle Corporation 1.7.0_25 (64-bit)/cpus=2,threads=1,free=23074440,total=44408832
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestDocumentObjectBinder, QueryResponseTest, TestUpdateRequestCodec, MultiCoreExampleJettyTest, LargeVolumeEmbeddedTest, TestBatchUpdate]
[junit4:junit4] Completed in 207.94s, 3 tests, 1 error <<< FAILURES!

[...truncated 131 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\build.xml:392: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\build.xml:372: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\build.xml:39: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build.xml:194: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\common-build.xml:437: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\lucene\common-build.xml:1248: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\lucene\common-build.xml:892: There were test failures: 46 suites, 272 tests, 1 error, 1 ignored (1 assumption)

Total time: 77 minutes 15 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0_25 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure