You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2013/11/28 11:48:20 UTC

[JENKINS] Lucene-Solr-trunk-Windows (32bit/jdk1.7.0_45) - Build # 3515 - Still Failing!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/3515/
Java: 32bit/jdk1.7.0_45 -server -XX:+UseG1GC

1 tests failed.
REGRESSION:  org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.testDistribSearch

Error Message:
There were expected update fails expected:<0> but was:<1>

Stack Trace:
java.lang.AssertionError: There were expected update fails expected:<0> but was:<1>
	at __randomizedtesting.SeedInfo.seed([F8729A664875CF4F:7994147E3F2AAF73]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.doTest(ChaosMonkeyNothingIsSafeTest.java:191)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
	at sun.reflect.GeneratedMethodAccessor46.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:744)




Build Log:
[...truncated 10735 lines...]
   [junit4] Suite: org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest
   [junit4]   2> 2928682 T8598 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
   [junit4]   2> 2928689 T8598 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> Creating dataDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\solrtest-ChaosMonkeyNothingIsSafeTest-1385635050266
   [junit4]   2> 2928691 T8598 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 2928692 T8599 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 2928771 T8598 oasc.ZkTestServer.run start zk server on port:59985
   [junit4]   2> 2928773 T8598 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2928781 T8605 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ebd27d name:ZooKeeperConnection Watcher:127.0.0.1:59985 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2928782 T8598 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2928782 T8598 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 2928788 T8600 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1429e48f3220000, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 
   [junit4]   2> 2928791 T8598 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2928793 T8607 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e9362b name:ZooKeeperConnection Watcher:127.0.0.1:59985/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2928794 T8598 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2928794 T8598 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 2928799 T8598 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 2928802 T8598 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 2928806 T8598 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 2928812 T8598 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 2928813 T8598 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 2928820 T8598 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 2928820 T8598 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 2928825 T8598 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2928826 T8598 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2928830 T8598 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 2928830 T8598 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 2928834 T8598 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 2928835 T8598 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 2928839 T8598 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 2928840 T8598 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 2928844 T8598 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 2928845 T8598 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 2928848 T8598 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2928848 T8598 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2928853 T8598 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2928854 T8598 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2928858 T8598 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 2928859 T8598 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 2928862 T8598 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test-files\solr\collection1\conf\synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 2928862 T8598 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 2928869 T8600 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1429e48f3220001, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 
   [junit4]   2> 2929234 T8598 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 2929243 T8598 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:59992
   [junit4]   2> 2929244 T8598 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 2929244 T8598 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2929245 T8598 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: .\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1385635050445
   [junit4]   2> 2929245 T8598 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1385635050445\'
   [junit4]   2> 2929274 T8598 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1385635050445\solr.xml
   [junit4]   2> 2929345 T8598 oasc.CoreContainer.<init> New CoreContainer 12608973
   [junit4]   2> 2929346 T8598 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1385635050445\]
   [junit4]   2> 2929347 T8598 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 2929347 T8598 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 2929348 T8598 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 2929348 T8598 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 2929349 T8598 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 2929349 T8598 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 2929349 T8598 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 2929350 T8598 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 2929350 T8598 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 2929358 T8598 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2929359 T8598 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2929360 T8598 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 2929360 T8598 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:59985/solr
   [junit4]   2> 2929360 T8598 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 2929363 T8598 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2929370 T8618 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@67ee45 name:ZooKeeperConnection Watcher:127.0.0.1:59985 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2929371 T8598 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2929374 T8598 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2929377 T8620 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a3dc74 name:ZooKeeperConnection Watcher:127.0.0.1:59985/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2929377 T8598 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2929379 T8598 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 2929383 T8598 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 2929386 T8598 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 2929389 T8598 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59992_
   [junit4]   2> 2929390 T8598 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59992_
   [junit4]   2> 2929393 T8598 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 2929403 T8598 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 2929407 T8598 oasc.Overseer.start Overseer (id=90808978654756867-127.0.0.1:59992_-n_0000000000) starting
   [junit4]   2> 2929417 T8598 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 2929423 T8622 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 2929424 T8598 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 2929428 T8598 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 2929431 T8598 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2929435 T8621 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 2929440 T8623 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 2929441 T8623 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 2929442 T8623 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 2929442 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2929445 T8621 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2929446 T8621 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:59992",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:59992_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 2929446 T8621 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
   [junit4]   2> 2929446 T8621 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 2929449 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2929451 T8620 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2930228 T8623 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2930228 T8623 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: .\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1385635050445\collection1
   [junit4]   2> 2930229 T8623 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
   [junit4]   2> 2930229 T8623 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 2930229 T8623 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 2930231 T8623 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1385635050445\collection1\'
   [junit4]   2> 2930233 T8623 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1385635050445/collection1/lib/classes/' to classloader
   [junit4]   2> 2930234 T8623 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1385635050445/collection1/lib/README' to classloader
   [junit4]   2> 2930290 T8623 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 2930341 T8623 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2930343 T8623 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 2930351 T8623 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 2930887 T8623 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2930894 T8623 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 2930898 T8623 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 2930917 T8623 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2930922 T8623 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2930927 T8623 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2930929 T8623 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 2930929 T8623 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 2930929 T8623 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2930930 T8623 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 2930930 T8623 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 2930931 T8623 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 2930931 T8623 oasc.SolrCore.<init> [collection1] Opening new SolrCore at .\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1385635050445\collection1\, dataDir=.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385635050265/control/data\
   [junit4]   2> 2930931 T8623 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@c2c1a8
   [junit4]   2> 2930932 T8623 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385635050265\control\data
   [junit4]   2> 2930933 T8623 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385635050265/control/data\index/
   [junit4]   2> 2930934 T8623 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385635050265\control\data\index' doesn't exist. Creating new index...
   [junit4]   2> 2930934 T8623 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385635050265\control\data\index
   [junit4]   2> 2930934 T8623 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=77.0498046875, floorSegmentMB=0.5947265625, forceMergeDeletesPctAllowed=6.166250316508858, segmentsPerTier=34.0, maxCFSSegmentSizeMB=0.2177734375, noCFSRatio=0.36120918980099437
   [junit4]   2> 2930938 T8623 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385635050265\control\data\index,segFN=segments_1,generation=1}
   [junit4]   2> 2930939 T8623 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2930942 T8623 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2930943 T8623 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2930943 T8623 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2930943 T8623 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2930943 T8623 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2930943 T8623 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2930944 T8623 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2930944 T8623 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2930944 T8623 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2930945 T8623 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 2930945 T8623 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 2930945 T8623 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 2930945 T8623 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 2930947 T8623 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 2930947 T8623 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 2930948 T8623 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 2930952 T8623 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2930957 T8623 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 2930957 T8623 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 2930957 T8623 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=29, maxMergeAtOnceExplicit=10, maxMergedSegmentMB=51.42578125, floorSegmentMB=1.388671875, forceMergeDeletesPctAllowed=1.0288149846681915, segmentsPerTier=23.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 2930957 T8623 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385635050265\control\data\index,segFN=segments_1,generation=1}
   [junit4]   2> 2930957 T8623 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2930957 T8623 oass.SolrIndexSearcher.<init> Opening Searcher@1753e39 main
   [junit4]   2> 2930962 T8624 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1753e39 main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 2930964 T8623 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 2930964 T8623 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:59992 collection:control_collection shard:shard1
   [junit4]   2> 2930964 T8623 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 2930972 T8623 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 2930975 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2930976 T8623 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 2930976 T8623 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C2489 name=collection1 org.apache.solr.core.SolrCore@17c5060 url=http://127.0.0.1:59992/collection1 node=127.0.0.1:59992_ C2489_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:59992, core=collection1, node_name=127.0.0.1:59992_}
   [junit4]   2> 2930976 T8623 C2489 P59992 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:59992/collection1/
   [junit4]   2> 2930976 T8623 C2489 P59992 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 2930976 T8623 C2489 P59992 oasc.SyncStrategy.syncToMe http://127.0.0.1:59992/collection1/ has no replicas
   [junit4]   2> 2930977 T8623 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:59992/collection1/ shard1
   [junit4]   2> 2930978 T8623 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 2930978 T8621 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2930983 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2930984 T8620 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2930990 T8621 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2930992 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2931067 T8620 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2931097 T8623 oasc.ZkController.register We are http://127.0.0.1:59992/collection1/ and leader is http://127.0.0.1:59992/collection1/
   [junit4]   2> 2931098 T8623 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:59992
   [junit4]   2> 2931098 T8623 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 2931098 T8623 oasc.ZkController.publish publishing core=collection1 state=active
   [junit4]   2> 2931098 T8623 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 2931100 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2931100 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2931100 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2931101 T8623 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2931102 T8598 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
   [junit4]   2> 2931103 T8598 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 2931105 T8621 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2931106 T8598 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2931107 T8621 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:59992",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:59992_",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 2931109 T8627 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@68c578 name:ZooKeeperConnection Watcher:127.0.0.1:59985/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2931109 T8598 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2931110 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2931110 T8598 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2931114 T8598 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 2931186 T8627 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2931187 T8620 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2931450 T8598 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 2931453 T8598 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:60004
   [junit4]   2> 2931455 T8598 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 2931455 T8598 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2931455 T8598 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1385635052690
   [junit4]   2> 2931455 T8598 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1385635052690\'
   [junit4]   2> 2931481 T8598 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1385635052690\solr.xml
   [junit4]   2> 2931542 T8598 oasc.CoreContainer.<init> New CoreContainer 9699407
   [junit4]   2> 2931542 T8598 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1385635052690\]
   [junit4]   2> 2931545 T8598 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 2931545 T8598 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 2931545 T8598 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 2931545 T8598 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 2931545 T8598 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 2931545 T8598 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 2931545 T8598 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 2931545 T8598 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 2931545 T8598 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 2931551 T8598 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2931551 T8598 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2931551 T8598 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 2931554 T8598 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:59985/solr
   [junit4]   2> 2931554 T8598 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 2931555 T8598 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2931561 T8638 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1cdddac name:ZooKeeperConnection Watcher:127.0.0.1:59985 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2931561 T8598 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2931563 T8600 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1429e48f3220005, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 
   [junit4]   2> 2931565 T8598 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2931567 T8640 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1846ac1 name:ZooKeeperConnection Watcher:127.0.0.1:59985/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2931568 T8598 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2931573 T8598 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2932294 T8598 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60004_
   [junit4]   2> 2932296 T8598 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:60004_
   [junit4]   2> 2932299 T8627 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 2932299 T8620 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 2932299 T8640 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 2932308 T8641 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 2932310 T8641 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 2932312 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2932312 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2932312 T8641 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 2932312 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2932315 T8621 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2932315 T8621 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:60004",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:60004_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 2932315 T8621 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
   [junit4]   2> 2932316 T8621 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 2932319 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2932408 T8620 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2932408 T8640 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2932408 T8627 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2933031 T8641 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2933031 T8641 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1385635052690\collection1
   [junit4]   2> 2933031 T8641 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 2933032 T8641 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 2933032 T8641 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 2933034 T8641 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1385635052690\collection1\'
   [junit4]   2> 2933035 T8641 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1385635052690/collection1/lib/classes/' to classloader
   [junit4]   2> 2933036 T8641 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1385635052690/collection1/lib/README' to classloader
   [junit4]   2> 2933090 T8641 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 2933144 T8641 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2933146 T8641 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 2933154 T8641 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 2933682 T8641 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2933689 T8641 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 2933691 T8641 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 2933710 T8641 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2933715 T8641 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2933721 T8641 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2933722 T8641 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 2933722 T8641 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 2933722 T8641 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2933724 T8641 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 2933724 T8641 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 2933724 T8641 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 2933724 T8641 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1385635052690\collection1\, dataDir=.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385635050265/jetty1\
   [junit4]   2> 2933725 T8641 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@c2c1a8
   [junit4]   2> 2933726 T8641 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385635050265\jetty1
   [junit4]   2> 2933727 T8641 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385635050265/jetty1\index/
   [junit4]   2> 2933727 T8641 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385635050265\jetty1\index' doesn't exist. Creating new index...
   [junit4]   2> 2933727 T8641 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385635050265\jetty1\index
   [junit4]   2> 2933728 T8641 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=77.0498046875, floorSegmentMB=0.5947265625, forceMergeDeletesPctAllowed=6.166250316508858, segmentsPerTier=34.0, maxCFSSegmentSizeMB=0.2177734375, noCFSRatio=0.36120918980099437
   [junit4]   2> 2933732 T8641 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385635050265\jetty1\index,segFN=segments_1,generation=1}
   [junit4]   2> 2933732 T8641 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2933736 T8641 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2933736 T8641 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2933737 T8641 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2933737 T8641 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2933737 T8641 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2933738 T8641 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2933738 T8641 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2933738 T8641 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2933738 T8641 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2933739 T8641 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 2933740 T8641 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 2933740 T8641 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 2933740 T8641 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 2933741 T8641 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 2933742 T8641 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 2933742 T8641 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 2933750 T8641 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2933755 T8641 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 2933755 T8641 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 2933755 T8641 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=29, maxMergeAtOnceExplicit=10, maxMergedSegmentMB=51.42578125, floorSegmentMB=1.388671875, forceMergeDeletesPctAllowed=1.0288149846681915, segmentsPerTier=23.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 2933757 T8641 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385635050265\jetty1\index,segFN=segments_1,generation=1}
   [junit4]   2> 2933757 T8641 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2933757 T8641 oass.SolrIndexSearcher.<init> Opening Searcher@15468f4 main
   [junit4]   2> 2933760 T8642 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@15468f4 main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 2933763 T8641 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 2933764 T8641 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:60004 collection:collection1 shard:shard2
   [junit4]   2> 2933764 T8641 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
   [junit4]   2> 2933773 T8641 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
   [junit4]   2> 2933776 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2933776 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2933776 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2933777 T8641 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 2933777 T8641 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C2490 name=collection1 org.apache.solr.core.SolrCore@165b001 url=http://127.0.0.1:60004/collection1 node=127.0.0.1:60004_ C2490_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:60004, core=collection1, node_name=127.0.0.1:60004_}
   [junit4]   2> 2933777 T8641 C2490 P60004 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:60004/collection1/
   [junit4]   2> 2933777 T8641 C2490 P60004 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 2933777 T8641 C2490 P60004 oasc.SyncStrategy.syncToMe http://127.0.0.1:60004/collection1/ has no replicas
   [junit4]   2> 2933777 T8641 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:60004/collection1/ shard2
   [junit4]   2> 2933777 T8641 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
   [junit4]   2> 2933780 T8621 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2933784 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2933786 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2933790 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2933865 T8627 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2933865 T8620 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2933865 T8640 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2933896 T8641 oasc.ZkController.register We are http://127.0.0.1:60004/collection1/ and leader is http://127.0.0.1:60004/collection1/
   [junit4]   2> 2933896 T8641 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:60004
   [junit4]   2> 2933896 T8641 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 2933897 T8641 oasc.ZkController.publish publishing core=collection1 state=active
   [junit4]   2> 2933897 T8641 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 2933898 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2933898 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2933899 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2933899 T8641 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2933900 T8598 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
   [junit4]   2> 2933900 T8598 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 2933900 T8621 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2933902 T8621 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:60004",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:60004_",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 2933907 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2933982 T8627 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2933985 T8620 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2933985 T8640 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2934232 T8598 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 2934235 T8598 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:60013
   [junit4]   2> 2934236 T8598 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 2934236 T8598 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2934236 T8598 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: .\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1385635055478
   [junit4]   2> 2934236 T8598 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1385635055478\'
   [junit4]   2> 2934262 T8598 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1385635055478\solr.xml
   [junit4]   2> 2934322 T8598 oasc.CoreContainer.<init> New CoreContainer 20611731
   [junit4]   2> 2934322 T8598 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1385635055478\]
   [junit4]   2> 2934324 T8598 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 2934324 T8598 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 2934324 T8598 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 2934324 T8598 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 2934324 T8598 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 2934324 T8598 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 2934324 T8598 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 2934326 T8598 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 2934326 T8598 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 2934332 T8598 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2934332 T8598 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2934332 T8598 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 2934332 T8598 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:59985/solr
   [junit4]   2> 2934334 T8598 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 2934335 T8598 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2934340 T8654 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1277c85 name:ZooKeeperConnection Watcher:127.0.0.1:59985 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2934341 T8598 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2934347 T8598 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2934349 T8656 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d72d02 name:ZooKeeperConnection Watcher:127.0.0.1:59985/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2934349 T8598 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2934354 T8598 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2935075 T8598 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60013_
   [junit4]   2> 2935077 T8598 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:60013_
   [junit4]   2> 2935080 T8627 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 2935081 T8656 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 2935082 T8620 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 2935082 T8640 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 2935089 T8657 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 2935090 T8657 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 2935091 T8657 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 2935091 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2935092 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2935092 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2935094 T8621 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2935095 T8621 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:60013",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:60013_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 2935095 T8621 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
   [junit4]   2> 2935095 T8621 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 2935098 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2935173 T8620 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 2935173 T8640 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 2935173 T8627 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 2935173 T8656 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 2935810 T8657 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2935810 T8657 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: .\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1385635055478\collection1
   [junit4]   2> 2935810 T8657 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 2935811 T8657 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 2935811 T8657 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 2935813 T8657 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1385635055478\collection1\'
   [junit4]   2> 2935814 T8657 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1385635055478/collection1/lib/classes/' to classloader
   [junit4]   2> 2935815 T8657 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1385635055478/collection1/lib/README' to classloader
   [junit4]   2> 2935869 T8657 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 2935920 T8657 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2935922 T8657 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 2935930 T8657 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 2936463 T8657 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2936472 T8657 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 2936475 T8657 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 2936490 T8657 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2936494 T8657 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2936498 T8657 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2936501 T8657 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 2936501 T8657 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 2936501 T8657 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2936502 T8657 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 2936502 T8657 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 2936502 T8657 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 2936503 T8657 oasc.SolrCore.<init> [collection1] Opening new SolrCore at .\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1385635055478\collection1\, dataDir=.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385635050265/jetty2\
   [junit4]   2> 2936503 T8657 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@c2c1a8
   [junit4]   2> 2936503 T8657 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385635050265\jetty2
   [junit4]   2> 2936503 T8657 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385635050265/jetty2\index/
   [junit4]   2> 2936505 T8657 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385635050265\jetty2\index' doesn't exist. Creating new index...
   [junit4]   2> 2936505 T8657 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385635050265\jetty2\index
   [junit4]   2> 2936505 T8657 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=77.0498046875, floorSegmentMB=0.5947265625, forceMergeDeletesPctAllowed=6.166250316508858, segmentsPerTier=34.0, maxCFSSegmentSizeMB=0.2177734375, noCFSRatio=0.36120918980099437
   [junit4]   2> 2936509 T8657 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385635050265\jetty2\index,segFN=segments_1,generation=1}
   [junit4]   2> 2936509 T8657 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2936513 T8657 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2936513 T8657 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2936514 T8657 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2936514 T8657 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2936514 T8657 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2936515 T8657 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2936515 T8657 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2936515 T8657 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2936515 T8657 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2936516 T8657 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 2936517 T8657 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 2936517 T8657 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 2936517 T8657 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 2936518 T8657 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 2936518 T8657 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 2936519 T8657 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 2936526 T8657 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2936531 T8657 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 2936531 T8657 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 2936533 T8657 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=29, maxMergeAtOnceExplicit=10, maxMergedSegmentMB=51.42578125, floorSegmentMB=1.388671875, forceMergeDeletesPctAllowed=1.0288149846681915, segmentsPerTier=23.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 2936534 T8657 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385635050265\jetty2\index,segFN=segments_1,generation=1}
   [junit4]   2> 2936534 T8657 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2936534 T8657 oass.SolrIndexSearcher.<init> Opening Searcher@14795ec main
   [junit4]   2> 2936542 T8657 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 2936542 T8657 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:60013 collection:collection1 shard:shard1
   [junit4]   2> 2936543 T8657 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 2936546 T8658 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@14795ec main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 2936550 T8657 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 2936553 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2936553 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2936553 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2936554 T8657 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 2936554 T8657 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C2491 name=collection1 org.apache.solr.core.SolrCore@c19934 url=http://127.0.0.1:60013/collection1 node=127.0.0.1:60013_ C2491_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:60013, core=collection1, node_name=127.0.0.1:60013_}
   [junit4]   2> 2936554 T8657 C2491 P60013 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:60013/collection1/
   [junit4]   2> 2936554 T8657 C2491 P60013 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 2936554 T8657 C2491 P60013 oasc.SyncStrategy.syncToMe http://127.0.0.1:60013/collection1/ has no replicas
   [junit4]   2> 2936554 T8657 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:60013/collection1/ shard1
   [junit4]   2> 2936554 T8657 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 2936557 T8621 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2936564 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2936566 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2936570 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2936645 T8656 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 2936646 T8627 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 2936646 T8620 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 2936646 T8640 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 2936678 T8657 oasc.ZkController.register We are http://127.0.0.1:60013/collection1/ and leader is http://127.0.0.1:60013/collection1/
   [junit4]   2> 2936679 T8657 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:60013
   [junit4]   2> 2936679 T8657 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 2936679 T8657 oasc.ZkController.publish publishing core=collection1 state=active
   [junit4]   2> 2936679 T8657 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 2936681 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2936681 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2936681 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2936681 T8657 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2936683 T8621 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2936683 T8598 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
   [junit4]   2> 2936683 T8598 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 2936684 T8621 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:60013",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:60013_",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node2"}
   [junit4]   2> 2936688 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2936763 T8620 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 2936763 T8640 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 2936763 T8656 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 2936763 T8627 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 2937014 T8598 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 2937018 T8598 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:60022
   [junit4]   2> 2937020 T8598 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 2937020 T8598 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2937020 T8598 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1385635058261
   [junit4]   2> 2937020 T8598 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1385635058261\'
   [junit4]   2> 2937046 T8598 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1385635058261\solr.xml
   [junit4]   2> 2937107 T8598 oasc.CoreContainer.<init> New CoreContainer 23050280
   [junit4]   2> 2937107 T8598 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1385635058261\]
   [junit4]   2> 2937109 T8598 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 2937109 T8598 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 2937109 T8598 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 2937109 T8598 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 2937109 T8598 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 2937109 T8598 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 2937109 T8598 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 2937111 T8598 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 2937112 T8598 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 2937117 T8598 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2937117 T8598 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2937117 T8598 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 2937117 T8598 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:59985/solr
   [junit4]   2> 2937119 T8598 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 2937122 T8598 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2937129 T8670 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1caaba name:ZooKeeperConnection Watcher:127.0.0.1:59985 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2937130 T8598 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2937135 T8598 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2937138 T8672 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@10cdd7c name:ZooKeeperConnection Watcher:127.0.0.1:59985/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2937138 T8598 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2937144 T8598 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2937808 T8598 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60022_
   [junit4]   2> 2937811 T8598 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:60022_
   [junit4]   2> 2937814 T8620 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 2937814 T8627 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 2937814 T8640 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 2937814 T8672 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 2937814 T8656 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 2937823 T8673 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 2937823 T8673 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 2937826 T8673 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 2937826 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2937826 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2937826 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2937828 T8621 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2937828 T8621 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:60022",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:60022_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 2937829 T8621 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
   [junit4]   2> 2937829 T8621 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 2937832 T8620 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2937900 T8627 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 2937900 T8640 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 2937900 T8620 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 2937900 T8672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 2937900 T8656 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 2938472 T8673 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2938472 T8673 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1385635058261\collection1
   [junit4]   2> 2938472 T8673 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 2938473 T8673 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 2938473 T8673 oascc.ZkStateReader.readConfigName Load collection config from:/collectio

[...truncated too long message...]

:
   [junit4]   1>      
   [junit4]   1> 
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ChaosMonkeyNothingIsSafeTest -Dtests.method=testDistribSearch -Dtests.seed=F8729A664875CF4F -Dtests.slow=true -Dtests.locale=lv_LV -Dtests.timezone=Etc/GMT+11 -Dtests.file.encoding=ISO-8859-1
   [junit4] FAILURE 52.7s | ChaosMonkeyNothingIsSafeTest.testDistribSearch <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: There were expected update fails expected:<0> but was:<1>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F8729A664875CF4F:7994147E3F2AAF73]:0)
   [junit4]    > 	at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.doTest(ChaosMonkeyNothingIsSafeTest.java:191)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 2981409 T8598 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: test params are: codec=Lucene46: {timestamp=Pulsing41(freqCutoff=9 minBlockSize=95 maxBlockSize=250), other_tl1=PostingsFormat(name=Memory doPackFST= false), _version_=PostingsFormat(name=Memory doPackFST= false), range_facet_sl=TestBloomFilteredLucene41Postings(BloomFilteringPostingsFormat(Lucene41(blocksize=128))), text=PostingsFormat(name=FSTOrdPulsing41), id=TestBloomFilteredLucene41Postings(BloomFilteringPostingsFormat(Lucene41(blocksize=128))), a_si=PostingsFormat(name=Memory doPackFST= false), range_facet_l=Pulsing41(freqCutoff=9 minBlockSize=95 maxBlockSize=250), multiDefault=Pulsing41(freqCutoff=9 minBlockSize=95 maxBlockSize=250), range_facet_si=Pulsing41(freqCutoff=9 minBlockSize=95 maxBlockSize=250), intDefault=PostingsFormat(name=Memory doPackFST= false), a_t=Pulsing41(freqCutoff=9 minBlockSize=95 maxBlockSize=250)}, docValues:{timestamp=DocValuesFormat(name=Lucene45)}, sim=DefaultSimilarity, locale=lv_LV, timezone=Etc/GMT+11
   [junit4]   2> NOTE: Windows 7 6.1 x86/Oracle Corporation 1.7.0_45 (32-bit)/cpus=2,threads=3,free=295791008,total=536870912
   [junit4]   2> NOTE: All tests run in this JVM: [CoreAdminCreateDiscoverTest, TestExtendedDismaxParser, TestPerFieldSimilarity, RequestHandlersTest, TestClassNameShortening, TestXIncludeConfig, DeleteReplicaTest, CurrencyFieldXmlFileTest, HdfsDirectoryTest, InfoHandlerTest, TestUpdate, AddBlockUpdateTest, TestAtomicUpdateErrorCases, TestPartialUpdateDeduplication, HdfsUnloadDistributedZkTest, SuggesterWFSTTest, DistributedDebugComponentTest, OutputWriterTest, TestRandomMergePolicy, TestSchemaSimilarityResource, AnalysisAfterCoreReloadTest, TestLFUCache, StressHdfsTest, SpatialFilterTest, WordBreakSolrSpellCheckerTest, PreAnalyzedUpdateProcessorTest, BasicFunctionalityTest, NoCacheHeaderTest, DocumentBuilderTest, StatelessScriptUpdateProcessorFactoryTest, RequiredFieldsTest, TestIndexSearcher, HdfsSyncSliceTest, TestFunctionQuery, TestFastWriter, LeaderElectionIntegrationTest, SolrCoreCheckLockOnStartupTest, TestCodecSupport, TestFastLRUCache, QueryResultKeyTest, ZkControllerTest, TestCollapseQParserPlugin, EnumFieldTest, PeerSyncTest, TestBadConfig, TestRecovery, QueryParsingTest, SOLR749Test, SignatureUpdateProcessorFactoryTest, TestCSVLoader, SpellPossibilityIteratorTest, TestQueryUtils, TestStressReorder, SolrIndexSplitterTest, TestAddFieldRealTimeGet, BasicDistributedZk2Test, TestQueryTypes, BadComponentTest, ExternalFileFieldSortTest, SimpleFacetsTest, TestRangeQuery, TestDocSet, TestReversedWildcardFilterFactory, SystemInfoHandlerTest, SolrPluginUtilsTest, XsltUpdateRequestHandlerTest, ChaosMonkeySafeLeaderTest, AbstractAnalyticsFacetTest, HighlighterTest, TestIndexingPerformance, HdfsBasicDistributedZkTest, CacheHeaderTest, CoreMergeIndexesAdminHandlerTest, TestRecoveryHdfs, SampleTest, TestDistribDocBasedVersion, TermVectorComponentDistributedTest, TestReplicationHandler, MBeansHandlerTest, TestUtils, SuggesterTSTTest, JSONWriterTest, FullSolrCloudDistribCmdsTest, CoreContainerCoreInitFailuresTest, ZkNodePropsTest, TestLMJelinekMercerSimilarityFactory, TestSolrXml, SynonymTokenizerTest, TestSolrXmlPersistor, DirectUpdateHandlerTest, TestGroupingSearch, TestMergePolicyConfig, SchemaVersionSpecificBehaviorTest, SyncSliceTest, TestCollationFieldDocValues, SliceStateTest, DistanceFunctionTest, TestFuzzyAnalyzedSuggestions, SolrIndexConfigTest, NotRequiredUniqueKeyTest, TestNoOpRegenerator, CustomCollectionTest, SolrInfoMBeanTest, DistributedSuggesterComponentTest, TestRandomFaceting, TestConfig, BadIndexSchemaTest, BlockCacheTest, TestDynamicFieldResource, TestCSVResponseWriter, PrimUtilsTest, ResourceLoaderTest, DocValuesMultiTest, LukeRequestHandlerTest, TestDocBasedVersionConstraints, TestMaxScoreQueryParser, AbstractAnalyticsStatsTest, StatsComponentTest, TestCollationField, AddSchemaFieldsUpdateProcessorFactoryTest, SliceStateUpdateTest, TestPseudoReturnFields, TestWriterPerf, BasicZkTest, LoggingHandlerTest, TestElisionMultitermQuery, TestManagedSchemaFieldResource, FileUtilsTest, ShowFileRequestHandlerTest, TestNumberUtils, SolrRequestParserTest, TestSolrXmlPersistence, TestIBSimilarityFactory, RangeFacetTest, UniqFieldsUpdateProcessorFactoryTest, DefaultValueUpdateProcessorTest, DeleteShardTest, SuggesterFSTTest, TestSchemaVersionResource, NumericFieldsTest, TriLevelCompositeIdRoutingTest, TestSolrQueryParserResource, TestFieldTypeCollectionResource, TestBinaryField, TestPHPSerializedResponseWriter, TestRealTimeGet, TestLRUCache, MinimalSchemaTest, CurrencyFieldOpenExchangeTest, TestSolrCoreProperties, PrimitiveFieldTypeTest, DateMathParserTest, JsonLoaderTest, TestSort, CoreAdminHandlerTest, TestDFRSimilarityFactory, OverseerCollectionProcessorTest, DistributedTermsComponentTest, URLClassifyProcessorTest, FieldFacetExtrasTest, FieldMutatingUpdateProcessorTest, IndexSchemaRuntimeFieldTest, TestStressVersions, TestDistributedSearch, ParsingFieldUpdateProcessorsTest, TestDefaultSearchFieldResource, TestSolrDeletionPolicy2, FastVectorHighlighterTest, TestSolrQueryParserDefaultOperatorResource, BasicDistributedZkTest, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest, OpenCloseCoreStressTest, OverseerTest, ShardRoutingTest, RecoveryZkTest, ClusterStateUpdateTest, LeaderElectionTest, ZkSolrClientTest, TestZkChroot, TestRandomDVFaceting, ZkCLITest, ShardRoutingCustomTest, AutoCommitTest, TestDistributedGrouping, TestFaceting, TestHashPartitioner, DistributedSpellCheckComponentTest, TestJoin, TestMultiCoreConfBootstrap, TestReload, HardAutoCommitTest, TestCoreContainer, TestSolr4Spatial, SolrCoreTest, SpellCheckComponentTest, SolrCmdDistributorTest, QueryElevationComponentTest, ConvertedLegacyTest, TestFiltering, TestLazyCores, SoftAutoCommitTest, DistributedQueryElevationComponentTest, SimplePostToolTest, TestCoreDiscovery, SpellCheckCollatorTest, TestFoldingMultitermQuery, DocValuesTest, SuggesterTest, TestStressLucene, TestTrie, PolyFieldTest, TestWordDelimiterFilterFactory, QueryEqualityTest, DirectUpdateHandlerOptimizeTest, SortByFunctionTest, TestRemoteStreaming, TestSolrDeletionPolicy1, DebugComponentTest, IndexBasedSpellCheckerTest, TestSurroundQueryParser, DisMaxRequestHandlerTest, StandardRequestHandlerTest, DirectSolrSpellCheckerTest, DocumentAnalysisRequestHandlerTest, TestOmitPositions, FileBasedSpellCheckerTest, TermsComponentTest, XmlUpdateRequestHandlerTest, TestValueSourceCache, PathHierarchyTokenizerFactoryTest, TermVectorComponentTest, MoreLikeThisHandlerTest, FieldAnalysisRequestHandlerTest, TestSolrQueryParser, TestArbitraryIndexDir, RegexBoostProcessorTest, IndexSchemaTest, TestJmxIntegration, ReturnFieldsTest, UpdateRequestProcessorFactoryTest, TestAnalyzedSuggestions, BinaryUpdateRequestHandlerTest, PingRequestHandlerTest, CSVRequestHandlerTest, TestComponentsName, SearchHandlerTest, TestBinaryResponseWriter, HighlighterConfigTest, TestQuerySenderListener, UpdateParamsTest, AlternateDirectoryTest, TestSolrIndexConfig, TestQuerySenderNoQuery, CopyFieldTest, ResponseLogComponentTest, TestStressRecovery, MultiTermTest, TestSearchPerf, TestPostingsSolrHighlighter, DirectSolrConnectionTest, TestLuceneMatchVersion, TestPhraseSuggestions, TestCharFilters, EchoParamsTest, TestSweetSpotSimilarityFactory, TestLMDirichletSimilarityFactory, TestBM25SimilarityFactory, TestDefaultSimilarityFactory, LegacyHTMLStripCharFilterTest, TestJmxMonitoredMap, TimeZoneUtilsTest, TestFastOutputStream, ScriptEngineTest, OpenExchangeRatesOrgProviderTest, PluginInfoTest, ChaosMonkeyNothingIsSafeTest]
   [junit4] Completed in 52.80s, 1 test, 1 failure <<< FAILURES!

[...truncated 218 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:420: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:400: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:39: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-targets.xml:37: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build.xml:189: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\common-build.xml:491: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1275: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:908: There were test failures: 354 suites, 1566 tests, 1 failure, 39 ignored (11 assumptions)

Total time: 87 minutes 46 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.7.0_45 -server -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure