You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2014/08/06 01:38:58 UTC

[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4790 - Still Failing

Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4790/

2 tests failed.
REGRESSION:  org.apache.solr.cloud.OverseerTest.testOverseerFailure

Error Message:
KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election

Stack Trace:
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election
	at __randomizedtesting.SeedInfo.seed([BF3CFB0E35F7DE0:FFB4043F1FA92C1]:0)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:285)
	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:282)
	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:282)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:155)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:314)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
	at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:155)
	at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:660)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.MultiThreadedOCPTest.testDistribSearch

Error Message:
We have a failed SPLITSHARD task

Stack Trace:
java.lang.AssertionError: We have a failed SPLITSHARD task
	at __randomizedtesting.SeedInfo.seed([BF3CFB0E35F7DE0:8A1541A894001DDC]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.MultiThreadedOCPTest.testTaskExclusivity(MultiThreadedOCPTest.java:125)
	at org.apache.solr.cloud.MultiThreadedOCPTest.doTest(MultiThreadedOCPTest.java:71)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:865)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11379 lines...]
   [junit4] Suite: org.apache.solr.cloud.OverseerTest
   [junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerTest-BF3CFB0E35F7DE0-001/init-core-data-001
   [junit4]   2> 1856167 T3762 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (true)
   [junit4]   2> 1856168 T3762 oas.SolrTestCaseJ4.initCore ####initCore
   [junit4]   2> 1856168 T3762 oas.SolrTestCaseJ4.initCore ####initCore end
   [junit4]   2> 1856171 T3762 oas.SolrTestCaseJ4.setUp ###Starting testShardLeaderChange
   [junit4]   2> 1856171 T3762 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1856172 T3763 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 1856272 T3762 oasc.ZkTestServer.run start zk server on port:18946
   [junit4]   2> 1856273 T3762 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1856277 T3769 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@497deeb6 name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1856277 T3762 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1856278 T3762 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1856279 T3771 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d172e27 name:ZooKeeperConnection Watcher:127.0.0.1:18946 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1856279 T3762 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1856281 T3762 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1856282 T3773 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5b5214f6 name:ZooKeeperConnection Watcher:127.0.0.1:18946 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1856282 T3762 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1856282 T3762 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 1856284 T3762 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 1856287 T3774 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1856288 T3762 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1856288 T3776 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7b9d22c2 name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1856289 T3774 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1856289 T3778 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@51a20a18 name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1856289 T3762 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1856289 T3774 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 1856289 T3762 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 1856291 T3774 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 1856291 T3762 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 1856292 T3762 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1856294 T3774 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:18946_solr
   [junit4]   2> 1856294 T3774 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 1856294 T3762 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
   [junit4]   2> 1856295 T3774 oasc.Overseer.start Overseer (id=92227472330063875-127.0.0.1:18946_solr-n_0000000000) starting
   [junit4]   2> 1856297 T3762 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 1856297 T3774 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 1856299 T3774 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 1856301 T3774 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 1856303 T3774 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 1856305 T3774 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 1856308 T3774 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 1856311 T3781 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 1856311 T3774 oasc.OverseerTest$OverseerRestarter.run Killing overseer.
   [junit4]   2> 1856313 T3774 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1856314 T3783 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@46a416ea name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1856314 T3774 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1856315 T3781 oasc.OverseerCollectionProcessor.run ERROR Unable to prioritize overseer  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /roles.json
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:251)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:248)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:248)
   [junit4]   2> 	at org.apache.solr.cloud.OverseerCollectionProcessor.prioritizeOverseerNodes(OverseerCollectionProcessor.java:412)
   [junit4]   2> 	at org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProcessor.java:256)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 1856317 T3774 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:18946_solr
   [junit4]   2> 1856317 T3774 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 1856319 T3774 oasc.Overseer.start Overseer (id=92227472330063877-127.0.0.1:18946_solr-n_0000000001) starting
   [junit4]   2> 1856326 T3785 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 1856326 T3784 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 1856328 T3784 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1856328 T3784 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"state1",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "core_node_name":"node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "base_url":"http://node1/solr/"}
   [junit4]   2> 1856328 T3784 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1]
   [junit4]   2> 1856329 T3784 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 1856330 T3786 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1856331 T3779 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1856800 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1856806 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 1856810 T3786 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1856811 T3786 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1856812 T3784 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1856811 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1856815 T3784 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"state2",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "core_node_name":"node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "base_url":"http://node1/solr/"}
   [junit4]   2> 1856815 T3784 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
   [junit4]   2> 1856817 T3786 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1856817 T3762 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1856818 T3788 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4b928720 name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1856818 T3762 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1856819 T3762 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1856821 T3762 oascc.SolrZkClient.makePath makePath: /live_nodes/node2
   [junit4]   2> 1856825 T3786 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1856825 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1856826 T3784 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"state1",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "core_node_name":"node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "base_url":"http://node1/solr/"}
   [junit4]   2> 1856826 T3784 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
   [junit4]   2> 1856828 T3786 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1856829 T3762 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1856930 T3779 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1856930 T3789 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1857031 T3762 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1857033 T3786 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1857034 T3790 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1857034 T3791 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1857035 T3784 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1857036 T3784 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"state2",
   [junit4]   2> 	  "node_name":"node2",
   [junit4]   2> 	  "core":"core4",
   [junit4]   2> 	  "core_node_name":"node2",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "base_url":"http://node2/solr/"}
   [junit4]   2> 1857036 T3784 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
   [junit4]   2> 1857036 T3784 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 1857038 T3791 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1857139 T3779 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1857139 T3789 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1857533 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1857539 T3792 oasc.LeaderElector$ElectionWatcher.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:285)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:282)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:282)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:204)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 1857540 T3789 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 1857543 T3791 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1857543 T3790 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1857543 T3786 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1857544 T3784 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1857547 T3786 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1857649 T3789 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1859039 T3779 oasc.LeaderElector$ElectionWatcher.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:285)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:282)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:282)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:204)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 1859041 T3762 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1859043 T3762 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1859044 T3794 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5a27853f name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1859045 T3762 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1859046 T3762 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1859047 T3762 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
   [junit4]   2> 1859051 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1859051 T3791 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1859051 T3790 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1859051 T3786 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1859052 T3774 oasc.OverseerTest$OverseerRestarter.run Killing overseer.
   [junit4]   2> 1859054 T3774 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1859055 T3797 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@40ef5f7d name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1859055 T3774 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1859056 T3784 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:18946_solr
   [junit4]   2> 1859057 T3798 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:302)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:299)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:299)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:317)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:289)
   [junit4]   2> 
   [junit4]   2> 1859058 T3774 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:18946_solr
   [junit4]   2> 1859059 T3774 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 1859059 T3795 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 1859060 T3774 oasc.Overseer.start Overseer (id=92227472330063880-127.0.0.1:18946_solr-n_0000000002) starting
   [junit4]   2> 1859067 T3800 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 1859068 T3799 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 1859070 T3799 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1859070 T3799 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"state1",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "core_node_name":"node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "base_url":"http://node1/solr/"}
   [junit4]   2> 1859071 T3799 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
   [junit4]   2> 1859072 T3801 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1859073 T3795 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1859075 T3799 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1859077 T3801 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1859179 T3795 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1860558 T3789 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 1860561 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1860561 T3803 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1860561 T3802 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1860561 T3801 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1860563 T3799 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1860563 T3799 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"state2",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "core_node_name":"node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "base_url":"http://node1/solr/"}
   [junit4]   2> 1860564 T3799 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
   [junit4]   2> 1860565 T3762 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1860566 T3801 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1860567 T3805 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2faf0bab name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1860567 T3762 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1860568 T3762 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1860570 T3762 oascc.SolrZkClient.makePath makePath: /live_nodes/node2
   [junit4]   2> 1860573 T3801 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1860573 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1860574 T3799 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"state1",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "core_node_name":"node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "base_url":"http://node1/solr/"}
   [junit4]   2> 1860574 T3799 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
   [junit4]   2> 1860576 T3801 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1860578 T3762 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1860580 T3801 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1860580 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1860581 T3799 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"state2",
   [junit4]   2> 	  "node_name":"node2",
   [junit4]   2> 	  "core":"core4",
   [junit4]   2> 	  "core_node_name":"node2",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "base_url":"http://node2/solr/"}
   [junit4]   2> 1860581 T3799 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
   [junit4]   2> 1860583 T3801 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1860585 T3807 oasc.LeaderElector$ElectionWatcher.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:285)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:282)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:282)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:204)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 1860586 T3808 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 1860586 T3794 oaz.ClientCnxn$EventThread.processEvent ERROR Error while calling watcher  java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@39e61331 rejected from java.util.concurrent.ThreadPoolExecutor@2ff613a9[Shutting down, pool size = 2, active threads = 2, queued tasks = 0, completed tasks = 6]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
   [junit4]   2> 	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:201)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> 
   [junit4]   2> 1860590 T3801 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1860592 T3801 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1860694 T3808 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1862085 T3795 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 1862086 T3762 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1862088 T3762 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1862090 T3811 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@196ac902 name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1862090 T3762 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1862091 T3762 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1862093 T3762 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
   [junit4]   2> 1862096 T3774 oasc.OverseerTest$OverseerRestarter.run Killing overseer.
   [junit4]   2> 1862097 T3801 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1862097 T3803 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1862097 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1862097 T3802 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1862097 T3799 oasc.Overseer$ClusterStateUpdater.run WARN Solr cannot talk to ZK, exiting Overseer main queue loop org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/queue
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:285)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:282)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:282)
   [junit4]   2> 	at org.apache.solr.cloud.DistributedQueue.orderedChildren(DistributedQueue.java:99)
   [junit4]   2> 	at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:587)
   [junit4]   2> 	at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:560)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:217)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 1862098 T3774 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1862099 T3799 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:18946_solr
   [junit4]   2> 1862100 T3815 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:302)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:299)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:299)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:317)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:289)
   [junit4]   2> 
   [junit4]   2> 1862100 T3814 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5b2a743e name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1862101 T3774 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1862103 T3774 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:18946_solr
   [junit4]   2> 1862104 T3774 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 1862105 T3812 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 1862105 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1862105 T3774 oasc.Overseer.start Overseer (id=92227472330063883-127.0.0.1:18946_solr-n_0000000003) starting
   [junit4]   2> 1862110 T3762 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1862112 T3817 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@41c55b19 name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1862112 T3762 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1862113 T3762 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1862114 T3819 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 1862115 T3818 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 1862115 T3762 oascc.SolrZkClient.makePath makePath: /live_nodes/node2
   [junit4]   2> 1862116 T3818 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1862117 T3818 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"state1",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "core_node_name":"node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "base_url":"http://node1/solr/"}
   [junit4]   2> 1862118 T3818 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
   [junit4]   2> 1862118 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1862118 T3821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1862120 T3820 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1862120 T3812 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1862123 T3762 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1862123 T3818 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1862124 T3818 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"state2",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "core_node_name":"node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "base_url":"http://node1/solr/"}
   [junit4]   2> 1862124 T3818 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
   [junit4]   2> 1862125 T3821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1862128 T3821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1862128 T3818 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"state1",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "core_node_name":"node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "base_url":"http://node1/solr/"}
   [junit4]   2> 1862128 T3818 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
   [junit4]   2> 1862130 T3821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1862231 T3820 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1862231 T3812 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1862324 T3762 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1862326 T3821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1862326 T3822 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1862326 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1862326 T3823 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1862328 T3818 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1862329 T3818 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"state2",
   [junit4]   2> 	  "node_name":"node2",
   [junit4]   2> 	  "core":"core4",
   [junit4]   2> 	  "core_node_name":"node2",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "base_url":"http://node2/solr/"}
   [junit4]   2> 1862329 T3818 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
   [junit4]   2> 1862330 T3823 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1862332 T3824 oasc.LeaderElector$ElectionWatcher.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:285)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:282)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:282)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:204)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 1862332 T3820 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 1862336 T3823 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1862338 T3823 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1862440 T3820 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1863832 T3812 oasc.LeaderElector$ElectionWatcher.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:285)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:282)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:282)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:204)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 1863834 T3762 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1863836 T3762 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1863837 T3826 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@16902c1c name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1863837 T3762 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1863838 T3762 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1863840 T3762 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
   [junit4]   2> 1863844 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1863844 T3821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1863844 T3822 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1863844 T3823 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1863846 T3818 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1863847 T3818 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"state1",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "core_node_name":"node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "base_url":"http://node1/solr/"}
   [junit4]   2> 1863847 T3818 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
   [junit4]   2> 1863849 T3823 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1863850 T3827 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 1863851 T3823 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1863851 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1863852 T3818 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"state2",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "core_node_name":"node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "base_url":"http://node1/solr/"}
   [junit4]   2> 1863852 T3818 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
   [junit4]   2> 1863854 T3823 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1863856 T3762 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1863856 T3823 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1863857 T3829 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3c1bc032 name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1863857 T3762 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1863858 T3762 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1863860 T3762 oascc.SolrZkClient.makePath makePath: /live_nodes/node2
   [junit4]   2> 1863863 T3823 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1863863 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1863864 T3818 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"state1",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "core_node_name":"node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "base_url":"http://node1/solr/"}
   [junit4]   2> 1863865 T3818 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
   [junit4]   2> 1863866 T3823 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1863868 T3762 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1863931 T3774 oasc.OverseerTest$OverseerRestarter.run Killing overseer.
   [junit4]   2> 1863933 T3774 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1863934 T3832 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@703b1b63 name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1863934 T3774 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1863934 T3818 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:18946_solr
   [junit4]   2> 1863936 T3833 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:302)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:299)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:299)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:317)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:289)
   [junit4]   2> 
   [junit4]   2> 1863937 T3774 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:18946_solr
   [junit4]   2> 1863938 T3774 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 1863939 T3774 oasc.Overseer.start Overseer (id=92227472330063887-127.0.0.1:18946_solr-n_0000000004) starting
   [junit4]   2> 1863946 T3835 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 1863948 T3834 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1863948 T3834 oasc.Overseer$ClusterStateUpdater.run Replaying operations from work queue.
   [junit4]   2> 1863949 T3834 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"state1",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "core_node_name":"node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "base_url":"http://node1/solr/"}
   [junit4]   2> 1863949 T3834 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
   [junit4]   2> 1863950 T3827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1863950 T3830 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1863952 T3834 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"state2",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "core_node_name":"node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "base_url":"http://node1/solr/"}
   [junit4]   2> 1863952 T3834 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
   [junit4]   2> 1863953 T3830 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1863953 T3827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1863955 T3827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1863955 T3830 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1863957 T3834 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"state1",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "core_node_name":"node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "base_url":"http://node1/solr/"}
   [junit4]   2> 1863958 T3834 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
   [junit4]   2> 1863958 T3830 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1863958 T3827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1863960 T3834 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 1864070 T3762 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1864072 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1864072 T3836 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1864074 T3834 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1864074 T3834 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"state2",
   [junit4]   2> 	  "node_name":"node2",
   [junit4]   2> 	  "core":"core4",
   [junit4]   2> 	  "core_node_name":"node2",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "base_url":"http://node2/solr/"}
   [junit4]   2> 1864074 T3834 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
   [junit4]   2> 1864076 T3836 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1864077 T3837 oasc.LeaderElector$ElectionWatcher.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:285)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:282)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:282)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:204)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 1864078 T3830 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 1864081 T3836 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1864084 T3836 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1864185 T3830 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1865577 T3827 oasc.LeaderElector$ElectionWatcher.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:285)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:282)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:282)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:204)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 1865579 T3762 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1865581 T3762 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1865582 T3839 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6e2c5422 name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1865583 T3762 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1865584 T3762 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1865586 T3762 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
   [junit4]   2> 1865590 T3836 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1865590 T3841 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1865590 T3842 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1865590 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1865591 T3834 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1865591 T3834 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"state1",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "core_node_name":"node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "base_url":"http://node1/solr/"}
   [junit4]   2> 1865592 T3834 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
   [junit4]   2> 1865593 T3842 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1865596 T3840 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 1865599 T3842 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1865601 T3842 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1865662 T3774 oasc.OverseerTest$OverseerRestarter.run Killing overseer.
   [junit4]   2> 1865664 T3774 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1865666 T3844 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6b8fc314 name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1865666 T3774 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1865666 T3834 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:18946_solr
   [junit4]   2> 1865667 T3845 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:302)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:299)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:299)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:317)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:289)
   [junit4]   2> 
   [junit4]   2> 1865669 T3774 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:18946_solr
   [junit4]   2> 1865669 T3774 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 1865671 T3774 oasc.Overseer.start Overseer (id=92227472330063889-127.0.0.1:18946_solr-n_0000000005) starting
   [junit4]   2> 1865678 T3847 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 1865679 T3846 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1865680 T3846 oasc.Overseer$ClusterStateUpdater.run Replaying operations from work queue.
   [junit4]   2> 1865681 T3846 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"state1",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "core_node_name":"node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "base_url":"http://node1/solr/"}
   [junit4]   2> 1865681 T3846 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
   [junit4]   2> 1865682 T3840 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1865684 T3840 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1865685 T3846 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 1867095 T3830 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 1867098 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1867098 T3848 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1867100 T3846 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1867100 T3846 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"state2",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "core_node_name":"node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "base_url":"http://node1/solr/"}
   [junit4]   2> 1867101 T3846 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
   [junit4]   2> 1867102 T3762 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1867103 T3848 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1867104 T3850 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@64ddd902 name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1867104 T3762 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1867105 T3762 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1867107 T3762 oascc.SolrZkClient.makePath makePath: /live_nodes/node2
   [junit4]   2> 1867110 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1867110 T3848 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1867112 T3846 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"state1",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "core_node_name":"node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "base_url":"http://node1/solr/"}
   [junit4]   2> 1867112 T3846 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
   [junit4]   2> 1867114 T3848 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1867115 T3762 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1867117 T3848 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1867117 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1867118 T3846 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"state2",
   [junit4]   2> 	  "node_name":"node2",
   [junit4]   2> 	  "core":"core4",
   [junit4]   2> 	  "core_node_name":"node2",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "base_url":"http://node2/solr/"}
   [junit4]   2> 1867118 T3846 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
   [junit4]   2> 1867120 T3848 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1867122 T3852 oasc.LeaderElector$ElectionWatcher.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:285)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:282)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:282)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:204)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 1867123 T3853 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 1867126 T3848 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1867129 T3848 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1867230 T3853 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1868622 T3840 oasc.LeaderElector$ElectionWatcher.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:285)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:282)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:282)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:204)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at java.util.concurrent.ThreadPool

[...truncated too long message...]

ld/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-BF3CFB0E35F7DE0-001/tempDir-006/ocptest_shardsplit_shard1_replica1/data/index [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-BF3CFB0E35F7DE0-001/tempDir-006/ocptest_shardsplit_shard1_replica1/data/index;done=false>>]
   [junit4]   2> 3338878 T8658 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-BF3CFB0E35F7DE0-001/tempDir-006/ocptest_shardsplit_shard1_replica1/data/index
   [junit4]   2> 3338879 T10223 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:31309_
   [junit4]   2> 3338882 T10225 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:302)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:299)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:299)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:317)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:289)
   [junit4]   2> 
   [junit4]   2> 3340381 T10218 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 3340383 T8658 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
   [junit4]   2> 3342887 T8658 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
   [junit4]   2> 3342890 T8658 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:48964 48964
   [junit4]   2> 3343106 T8659 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:48964 48964
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=MultiThreadedOCPTest -Dtests.method=testDistribSearch -Dtests.seed=BF3CFB0E35F7DE0 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_CR -Dtests.timezone=US/East-Indiana -Dtests.file.encoding=ISO-8859-1
   [junit4] FAILURE 57.1s J0 | MultiThreadedOCPTest.testDistribSearch <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: We have a failed SPLITSHARD task
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([BF3CFB0E35F7DE0:8A1541A894001DDC]:0)
   [junit4]    > 	at org.apache.solr.cloud.MultiThreadedOCPTest.testTaskExclusivity(MultiThreadedOCPTest.java:125)
   [junit4]    > 	at org.apache.solr.cloud.MultiThreadedOCPTest.doTest(MultiThreadedOCPTest.java:71)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:865)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 3343129 T8658 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-BF3CFB0E35F7DE0-001
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=es_CR, timezone=US/East-Indiana
   [junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_60 (64-bit)/cpus=16,threads=1,free=255704496,total=536870912
   [junit4]   2> NOTE: All tests run in this JVM: [TestTrie, SolrCoreCheckLockOnStartupTest, TestRemoteStreaming, CurrencyFieldOpenExchangeTest, TestReplicationHandler, SolrXmlInZkTest, AnalyticsQueryTest, SuggesterWFSTTest, ZkControllerTest, FunctionTest, TestSolrXml, SolrIndexSplitterTest, SolrCmdDistributorTest, FieldFacetTest, AddBlockUpdateTest, PrimitiveFieldTypeTest, DistributedSpellCheckComponentTest, DisMaxRequestHandlerTest, TestClassNameShortening, HdfsSyncSliceTest, TestCloudManagedSchemaConcurrent, AssignTest, TestDocBasedVersionConstraints, RecoveryZkTest, PreAnalyzedFieldTest, TestManagedResource, TestTolerantSearch, PrimUtilsTest, SignatureUpdateProcessorFactoryTest, TestSort, BasicDistributedZkTest, TestComplexPhraseQParserPlugin, HdfsBasicDistributedZkTest, RegexBoostProcessorTest, TestWordDelimiterFilterFactory, TestHighlightDedupGrouping, XsltUpdateRequestHandlerTest, SimpleFacetsTest, HdfsBasicDistributedZk2Test, ExpressionTest, TestCollationFieldDocValues, TestDefaultSimilarityFactory, PeerSyncTest, TestSolrIndexConfig, DocumentBuilderTest, TestAddFieldRealTimeGet, TestManagedStopFilterFactory, SyncSliceTest, TestConfig, TestReloadAndDeleteDocs, TestNoOpRegenerator, DocExpirationUpdateProcessorFactoryTest, TestFileDictionaryLookup, HighlighterConfigTest, TestDefaultSearchFieldResource, DistributedIntervalFacetingTest, SpellingQueryConverterTest, AddSchemaFieldsUpdateProcessorFactoryTest, ChangedSchemaMergeTest, CoreAdminHandlerTest, LeaderElectionTest, HighlighterMaxOffsetTest, TestOverriddenPrefixQueryForCustomFieldType, SuggestComponentTest, ParsingFieldUpdateProcessorsTest, BadIndexSchemaTest, WordBreakSolrSpellCheckerTest, TestSolrCoreProperties, TestIntervalFaceting, QueryEqualityTest, TestRequestStatusCollectionAPI, TestManagedSchema, TestReplicationHandlerBackup, TestCoreDiscovery, DirectUpdateHandlerTest, PluginInfoTest, SpellCheckCollatorTest, HdfsCollectionsAPIDistributedZkTest, TestArbitraryIndexDir, AnalyticsMergeStrategyTest, ReturnFieldsTest, TestStressVersions, OpenExchangeRatesOrgProviderTest, IndexBasedSpellCheckerTest, ZkCLITest, ClusterStateUpdateTest, TestSchemaResource, TestHashPartitioner, TestSchemaVersionResource, ResponseLogComponentTest, OverseerTest, TestStressRecovery, TestSearchPerf, TestQuerySenderListener, TestComponentsName, DocValuesMultiTest, TestFieldTypeResource, ShardRoutingTest, OverseerStatusTest, XmlUpdateRequestHandlerTest, OverseerCollectionProcessorTest, CoreAdminCreateDiscoverTest, TestExceedMaxTermLength, BasicDistributedZk2Test, TestZkChroot, TestRandomDVFaceting, TestDistributedSearch, TestRangeQuery, TestSolr4Spatial, SolrCoreTest, StatsComponentTest, SpellCheckComponentTest, TestGroupingSearch, BasicFunctionalityTest, TestLazyCores, TestBadConfig, TestIndexSearcher, HighlighterTest, ShowFileRequestHandlerTest, SuggesterFSTTest, TestFoldingMultitermQuery, SuggesterTest, TestCSVLoader, NoCacheHeaderTest, TestSolrDeletionPolicy1, TestSurroundQueryParser, TestQueryUtils, StandardRequestHandlerTest, TestWriterPerf, TestReversedWildcardFilterFactory, TestQueryTypes, TermsComponentTest, TestValueSourceCache, TermVectorComponentTest, MoreLikeThisHandlerTest, FieldAnalysisRequestHandlerTest, IndexSchemaRuntimeFieldTest, SolrPluginUtilsTest, TestJmxIntegration, JSONWriterTest, QueryParsingTest, JsonLoaderTest, TestPHPSerializedResponseWriter, CSVRequestHandlerTest, TestLFUCache, AlternateDirectoryTest, TestMergePolicyConfig, MultiTermTest, TestDocSet, MinimalSchemaTest, TestFuzzyAnalyzedSuggestions, ExternalFileFieldSortTest, TestLuceneMatchVersion, SpellPossibilityIteratorTest, TestCodecSupport, TestSweetSpotSimilarityFactory, TestLMDirichletSimilarityFactory, TestIBSimilarityFactory, TestFastWriter, LegacyHTMLStripCharFilterTest, TimeZoneUtilsTest, URLClassifyProcessorTest, TestSystemIdResolver, DateFieldTest, DOMUtilTest, RAMDirectoryFactoryTest, ClusterStateTest, TestSolrJ, TestLRUCache, TestUtils, ZkNodePropsTest, FileUtilsTest, CircularListTest, TestRTGBase, TestCursorMarkWithoutUniqueKey, AbstractAnalyticsStatsTest, AbstractAnalyticsFacetTest, AliasIntegrationTest, CustomCollectionTest, DeleteInactiveReplicaTest, DistribDocExpirationUpdateProcessorTest, MultiThreadedOCPTest]
   [junit4] Completed on J0 in 57.11s, 1 test, 1 failure <<< FAILURES!

[...truncated 215 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:467: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:447: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:45: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:496: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1298: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:922: There were test failures: 422 suites, 1757 tests, 1 error, 1 failure, 53 ignored (19 assumptions)

Total time: 103 minutes 8 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-Tests-trunk-Java7 #4781
Archived 3 artifacts
Archive block size is 32768
Received 0 blocks and 238011058 bytes
Compression is 0.0%
Took 42 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4792 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4792/

1 tests failed.
FAILED:  org.apache.solr.client.solrj.embedded.SolrExampleStreamingTest.testExampleConfig

Error Message:
Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 500 {msg=posix_spawn is not a supported process launch mechanism on this platform.,trace=java.lang.Error: posix_spawn is not a supported process launch mechanism on this platform.  at java.lang.UNIXProcess$1.run(UNIXProcess.java:111)  at java.lang.UNIXProcess$1.run(UNIXProcess.java:93)  at java.security.AccessController.doPrivileged(Native Method)  at java.lang.UNIXProcess.&lt;clinit&gt;(UNIXProcess.java:91)  at java.lang.ProcessImpl.start(ProcessImpl.java:130)  at java.lang.ProcessBuilder.start(ProcessBuilder.java:1022)  at java.lang.Runtime.exec(Runtime.java:617)  at java.lang.Runtime.exec(Runtime.java:450)  at java.lang.Runtime.exec(Runtime.java:347)  at org.apache.solr.handler.admin.SystemInfoHandler.execute(SystemInfoHandler.java:220)  at org.apache.solr.handler.admin.SystemInfoHandler.getSystemInfo(SystemInfoHandler.java:176)  at org.apache.solr.handler.admin.SystemInfoHandler.handleRequestBody(SystemInfoHandler.java:97)  at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)  at org.apache.solr.handler.admin.InfoHandler.handleRequestBody(InfoHandler.java:88)  at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)  at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:732)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:275)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)  at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:137)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)  at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)  at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)  at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301)  at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077)  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)  at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)  at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)  at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)  at org.eclipse.jetty.server.Server.handle(Server.java:368)  at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)  at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)  at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)  at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)  at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)  at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)  at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)  at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)  at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)  at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)  at java.lang.Thread.run(Thread.java:745) ,code=500}</title> </head> <body> <h2>HTTP ERROR: 500</h2> <p>Problem accessing /solr/admin/info/system. Reason: <pre>    {msg=posix_spawn is not a supported process launch mechanism on this platform.,trace=java.lang.Error: posix_spawn is not a supported process launch mechanism on this platform.  at java.lang.UNIXProcess$1.run(UNIXProcess.java:111)  at java.lang.UNIXProcess$1.run(UNIXProcess.java:93)  at java.security.AccessController.doPrivileged(Native Method)  at java.lang.UNIXProcess.&lt;clinit&gt;(UNIXProcess.java:91)  at java.lang.ProcessImpl.start(ProcessImpl.java:130)  at java.lang.ProcessBuilder.start(ProcessBuilder.java:1022)  at java.lang.Runtime.exec(Runtime.java:617)  at java.lang.Runtime.exec(Runtime.java:450)  at java.lang.Runtime.exec(Runtime.java:347)  at org.apache.solr.handler.admin.SystemInfoHandler.execute(SystemInfoHandler.java:220)  at org.apache.solr.handler.admin.SystemInfoHandler.getSystemInfo(SystemInfoHandler.java:176)  at org.apache.solr.handler.admin.SystemInfoHandler.handleRequestBody(SystemInfoHandler.java:97)  at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)  at org.apache.solr.handler.admin.InfoHandler.handleRequestBody(InfoHandler.java:88)  at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)  at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:732)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:275)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)  at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:137)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)  at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)  at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)  at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301)  at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077)  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)  at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)  at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)  at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)  at org.eclipse.jetty.server.Server.handle(Server.java:368)  at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)  at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)  at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)  at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)  at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)  at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)  at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)  at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)  at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)  at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)  at java.lang.Thread.run(Thread.java:745) ,code=500}</pre></p> <hr /><i><small>Powered by Jetty://</small></i>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     </body> </html> 

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 500 {msg=posix_spawn is not a supported process launch mechanism on this platform.,trace=java.lang.Error: posix_spawn is not a supported process launch mechanism on this platform.
	at java.lang.UNIXProcess$1.run(UNIXProcess.java:111)
	at java.lang.UNIXProcess$1.run(UNIXProcess.java:93)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.lang.UNIXProcess.&lt;clinit&gt;(UNIXProcess.java:91)
	at java.lang.ProcessImpl.start(ProcessImpl.java:130)
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1022)
	at java.lang.Runtime.exec(Runtime.java:617)
	at java.lang.Runtime.exec(Runtime.java:450)
	at java.lang.Runtime.exec(Runtime.java:347)
	at org.apache.solr.handler.admin.SystemInfoHandler.execute(SystemInfoHandler.java:220)
	at org.apache.solr.handler.admin.SystemInfoHandler.getSystemInfo(SystemInfoHandler.java:176)
	at org.apache.solr.handler.admin.SystemInfoHandler.handleRequestBody(SystemInfoHandler.java:97)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
	at org.apache.solr.handler.admin.InfoHandler.handleRequestBody(InfoHandler.java:88)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
	at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:732)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:275)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:137)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
	at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
	at org.eclipse.jetty.server.Server.handle(Server.java:368)
	at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
	at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
	at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
	at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
	at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
	at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
	at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
	at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
	at java.lang.Thread.run(Thread.java:745)
,code=500}</title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /solr/admin/info/system. Reason:
<pre>    {msg=posix_spawn is not a supported process launch mechanism on this platform.,trace=java.lang.Error: posix_spawn is not a supported process launch mechanism on this platform.
	at java.lang.UNIXProcess$1.run(UNIXProcess.java:111)
	at java.lang.UNIXProcess$1.run(UNIXProcess.java:93)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.lang.UNIXProcess.&lt;clinit&gt;(UNIXProcess.java:91)
	at java.lang.ProcessImpl.start(ProcessImpl.java:130)
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1022)
	at java.lang.Runtime.exec(Runtime.java:617)
	at java.lang.Runtime.exec(Runtime.java:450)
	at java.lang.Runtime.exec(Runtime.java:347)
	at org.apache.solr.handler.admin.SystemInfoHandler.execute(SystemInfoHandler.java:220)
	at org.apache.solr.handler.admin.SystemInfoHandler.getSystemInfo(SystemInfoHandler.java:176)
	at org.apache.solr.handler.admin.SystemInfoHandler.handleRequestBody(SystemInfoHandler.java:97)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
	at org.apache.solr.handler.admin.InfoHandler.handleRequestBody(InfoHandler.java:88)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
	at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:732)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:275)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:137)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
	at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
	at org.eclipse.jetty.server.Server.handle(Server.java:368)
	at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
	at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
	at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
	at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
	at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
	at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
	at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
	at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
	at java.lang.Thread.run(Thread.java:745)
,code=500}</pre></p>
<hr /><i><small>Powered by Jetty://</small></i>
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
</body>
</html>

	at __randomizedtesting.SeedInfo.seed([25EEF9A8D20C97FB:92C3ED30150B7122]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:513)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
	at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:91)
	at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:301)
	at org.apache.solr.client.solrj.SolrExampleTests.testExampleConfig(SolrExampleTests.java:220)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 12578 lines...]
   [junit4] Suite: org.apache.solr.client.solrj.embedded.SolrExampleStreamingTest
   [junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/./temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/init-core-data-001
   [junit4]   2> 161427 T6286 oas.SolrTestCaseJ4.initCore ####initCore
   [junit4]   2> 161428 T6286 oas.SolrTestCaseJ4.initCore ####initCore end
   [junit4]   2> 161429 T6286 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 161434 T6286 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:59691
   [junit4]   2> 161434 T6286 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 161435 T6286 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 161435 T6286 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr
   [junit4]   2> 161436 T6286 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/'
   [junit4]   2> 161477 T6286 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/solr.xml
   [junit4]   2> 161547 T6286 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr
   [junit4]   2> 161547 T6286 oasc.CoreContainer.<init> New CoreContainer 761401314
   [junit4]   2> 161548 T6286 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/]
   [junit4]   2> 161549 T6286 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
   [junit4]   2> 161549 T6286 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
   [junit4]   2> 161550 T6286 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
   [junit4]   2> 161550 T6286 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 161551 T6286 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 161551 T6286 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 161551 T6286 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 161552 T6286 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 161552 T6286 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 161553 T6286 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=0&connTimeout=0&retry=false
   [junit4]   2> 161553 T6286 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 161554 T6286 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 161554 T6286 oasc.CoreContainer.load Host Name: 
   [junit4]   2> 161558 T6286 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr
   [junit4]   2> 161560 T6286 oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/collection1/
   [junit4]   2> 161572 T6286 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 161573 T6296 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/collection1/'
   [junit4]   2> 161624 T6296 oasc.SolrConfig.initLibs Adding specified lib dirs to ClassLoader
   [junit4]   2> 161625 T6296 oasc.SolrResourceLoader.addToClassLoader WARN No files added to classloader from lib: ../../../contrib/extraction/lib (resolved as: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/collection1/../../../contrib/extraction/lib).
   [junit4]   2> 161626 T6296 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/collection1/../../../dist).
   [junit4]   2> 161626 T6296 oasc.SolrResourceLoader.addToClassLoader WARN No files added to classloader from lib: ../../../contrib/clustering/lib/ (resolved as: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/collection1/../../../contrib/clustering/lib).
   [junit4]   2> 161627 T6296 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/collection1/../../../dist).
   [junit4]   2> 161628 T6296 oasc.SolrResourceLoader.addToClassLoader WARN No files added to classloader from lib: ../../../contrib/langid/lib/ (resolved as: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/collection1/../../../contrib/langid/lib).
   [junit4]   2> 161628 T6296 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/collection1/../../../dist).
   [junit4]   2> 161629 T6296 oasc.SolrResourceLoader.addToClassLoader WARN No files added to classloader from lib: ../../../contrib/velocity/lib (resolved as: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/collection1/../../../contrib/velocity/lib).
   [junit4]   2> 161629 T6296 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/collection1/../../../dist).
   [junit4]   2> 161693 T6296 oasu.SolrIndexConfig.<init> IndexWriter infoStream solr logging is enabled
   [junit4]   2> 161698 T6296 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
   [junit4]   2> 161791 T6296 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 161792 T6296 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 161840 T6296 oass.IndexSchema.readSchema [collection1] Schema name=example
   [junit4]   2> 162452 T6296 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 162454 T6296 oasr.RestManager$Registry.registerManagedResource Registered ManagedResource impl org.apache.solr.rest.schema.analysis.ManagedWordSetResource for path /schema/analysis/stopwords/english
   [junit4]   2> 162454 T6296 oasr.RestManager$Registry.registerManagedResource Registered ManagedResource impl org.apache.solr.rest.schema.analysis.ManagedSynonymFilterFactory$SynonymManager for path /schema/analysis/synonyms/english
   [junit4]   2> 162476 T6296 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 162482 T6296 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 162513 T6296 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from instancedir /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/collection1/
   [junit4]   2> 162513 T6296 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 162514 T6296 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/
   [junit4]   2> 162514 T6296 oasc.JmxMonitoredMap.<init> No JMX servers found, not exposing Solr information with JMX.
   [junit4]   2> 162514 T6296 oasc.SolrCore.initListeners [collection1] Added SolrEventListener for newSearcher: org.apache.solr.core.QuerySenderListener{queries=[]}
   [junit4]   2> 162514 T6296 oasc.SolrCore.initListeners [collection1] Added SolrEventListener for firstSearcher: org.apache.solr.core.QuerySenderListener{queries=[{q=static firstSearcher warming in solrconfig.xml}]}
   [junit4]   2> 162515 T6296 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001
   [junit4]   2> 162515 T6296 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/index/
   [junit4]   2> 162515 T6296 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/index' doesn't exist. Creating new index...
   [junit4]   2> 162516 T6296 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/index
   [junit4]   2> 162517 T6296 oasu.LoggingInfoStream.message [IFD][coreLoadExecutor-200-thread-1]: init: current segments file is "null"; deletionPolicy=org.apache.solr.core.IndexDeletionPolicyWrapper@44dd1ac2
   [junit4]   2> 162518 T6296 oasu.LoggingInfoStream.message [IFD][coreLoadExecutor-200-thread-1]: now checkpoint "" [0 segments ; isCommit = false]
   [junit4]   2> 162518 T6296 oasu.LoggingInfoStream.message [IFD][coreLoadExecutor-200-thread-1]: 0 msec to checkpoint
   [junit4]   2> 162518 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: init: create=true
   [junit4]   2> 162518 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: 
   [junit4]   2> 	dir=MockDirectoryWrapper(RAMDirectory@1c06dda5 lockFactory=NativeFSLockFactory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/index)
   [junit4]   2> 	index=
   [junit4]   2> 	version=5.0-SNAPSHOT
   [junit4]   2> 	matchVersion=LUCENE_5_0
   [junit4]   2> 	analyzer=null
   [junit4]   2> 	ramBufferSizeMB=100.0
   [junit4]   2> 	maxBufferedDocs=-1
   [junit4]   2> 	maxBufferedDeleteTerms=-1
   [junit4]   2> 	mergedSegmentWarmer=null
   [junit4]   2> 	delPolicy=org.apache.solr.core.IndexDeletionPolicyWrapper
   [junit4]   2> 	commit=null
   [junit4]   2> 	openMode=CREATE
   [junit4]   2> 	similarity=org.apache.lucene.search.similarities.DefaultSimilarity
   [junit4]   2> 	mergeScheduler=ConcurrentMergeScheduler: maxThreadCount=1, maxMergeCount=2, mergeThreadPriority=-1
   [junit4]   2> 	default WRITE_LOCK_TIMEOUT=1000
   [junit4]   2> 	writeLockTimeout=1000
   [junit4]   2> 	codec=Lucene49
   [junit4]   2> 	infoStream=org.apache.solr.update.LoggingInfoStream
   [junit4]   2> 	mergePolicy=[TieredMergePolicy: maxMergeAtOnce=10, maxMergeAtOnceExplicit=30, maxMergedSegmentMB=5120.0, floorSegmentMB=2.0, forceMergeDeletesPctAllowed=10.0, segmentsPerTier=10.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 	indexerThreadPool=org.apache.lucene.index.DocumentsWriterPerThreadPool@1007d9e0
   [junit4]   2> 	readerPooling=false
   [junit4]   2> 	perThreadHardLimitMB=1945
   [junit4]   2> 	useCompoundFile=false
   [junit4]   2> 	checkIntegrityAtMerge=false
   [junit4]   2> 	writer=org.apache.lucene.util.SetOnce@73710958
   [junit4]   2> 	
   [junit4]   2> 162519 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: now flush at shutdown
   [junit4]   2> 162519 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]:   start flush: applyAllDeletes=true
   [junit4]   2> 162519 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]:   index before flush 
   [junit4]   2> 162519 T6296 oasu.LoggingInfoStream.message [DW][coreLoadExecutor-200-thread-1]: startFullFlush
   [junit4]   2> 162519 T6296 oasu.LoggingInfoStream.message [DW][coreLoadExecutor-200-thread-1]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
   [junit4]   2> 162520 T6296 oasu.LoggingInfoStream.message [DW][coreLoadExecutor-200-thread-1]: coreLoadExecutor-200-thread-1 finishFullFlush success=true
   [junit4]   2> 162520 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: apply all deletes during flush
   [junit4]   2> 162520 T6296 oasu.LoggingInfoStream.message [BD][coreLoadExecutor-200-thread-1]: prune sis=null:  minGen=9223372036854775807 packetCount=0
   [junit4]   2> 162520 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: waitForMerges
   [junit4]   2> 162520 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: waitForMerges done
   [junit4]   2> 162521 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: commit: start
   [junit4]   2> 162521 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: commit: enter lock
   [junit4]   2> 162521 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: commit: now prepare
   [junit4]   2> 162521 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: prepareCommit: flush
   [junit4]   2> 162521 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]:   index before flush 
   [junit4]   2> 162522 T6296 oasu.LoggingInfoStream.message [DW][coreLoadExecutor-200-thread-1]: startFullFlush
   [junit4]   2> 162522 T6296 oasu.LoggingInfoStream.message [DW][coreLoadExecutor-200-thread-1]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
   [junit4]   2> 162522 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: apply all deletes during flush
   [junit4]   2> 162522 T6296 oasu.LoggingInfoStream.message [BD][coreLoadExecutor-200-thread-1]: prune sis=null:  minGen=9223372036854775807 packetCount=0
   [junit4]   2> 162522 T6296 oasu.LoggingInfoStream.message [DW][coreLoadExecutor-200-thread-1]: coreLoadExecutor-200-thread-1 finishFullFlush success=true
   [junit4]   2> 162523 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: startCommit(): start
   [junit4]   2> 162523 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: startCommit index= changeCount=1
   [junit4]   2> 162523 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: done all syncs: []
   [junit4]   2> 162523 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: commit: pendingCommit != null
   [junit4]   2> 162524 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: commit: wrote segments file "segments_1"
   [junit4]   2> 162524 T6296 oasu.LoggingInfoStream.message [IFD][coreLoadExecutor-200-thread-1]: now checkpoint "" [0 segments ; isCommit = true]
   [junit4]   2> 162524 T6296 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@1c06dda5 lockFactory=NativeFSLockFactory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/index),segFN=segments_1,generation=1}
   [junit4]   2> 162524 T6296 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 162525 T6296 oasu.LoggingInfoStream.message [IFD][coreLoadExecutor-200-thread-1]: 0 msec to checkpoint
   [junit4]   2> 162525 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: commit: took 3.6 msec
   [junit4]   2> 162525 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: commit: done
   [junit4]   2> 162525 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: rollback
   [junit4]   2> 162525 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: all running merges have aborted
   [junit4]   2> 162526 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: rollback: done finish merges
   [junit4]   2> 162526 T6296 oasu.LoggingInfoStream.message [DW][coreLoadExecutor-200-thread-1]: abort
   [junit4]   2> 162526 T6296 oasu.LoggingInfoStream.message [DW][coreLoadExecutor-200-thread-1]: done abort; abortedFiles=[] success=true
   [junit4]   2> 162526 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: rollback: infos=
   [junit4]   2> 162526 T6296 oasu.LoggingInfoStream.message [IFD][coreLoadExecutor-200-thread-1]: now checkpoint "" [0 segments ; isCommit = false]
   [junit4]   2> 162527 T6296 oasu.LoggingInfoStream.message [IFD][coreLoadExecutor-200-thread-1]: 0 msec to checkpoint
   [junit4]   2> 162527 T6296 oasc.SolrCore.initWriters created json: solr.JSONResponseWriter
   [junit4]   2> 162527 T6296 oasc.SolrCore.initWriters adding lazy queryResponseWriter: solr.VelocityResponseWriter
   [junit4]   2> 162527 T6296 oasc.SolrCore.initWriters created velocity: solr.VelocityResponseWriter
   [junit4]   2> 162528 T6296 oasc.SolrCore.initWriters created xslt: solr.XSLTResponseWriter
   [junit4]   2> 162528 T6296 oasr.XSLTResponseWriter.init xsltCacheLifetimeSeconds=5
   [junit4]   2> 162529 T6296 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 162529 T6296 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 162530 T6296 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 162530 T6296 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 162530 T6296 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 162530 T6296 oasc.RequestHandlers.initHandlersFromConfig created /select: solr.SearchHandler
   [junit4]   2> 162531 T6296 oasc.RequestHandlers.initHandlersFromConfig created /query: solr.SearchHandler
   [junit4]   2> 162531 T6296 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 162531 T6296 oasc.RequestHandlers.initHandlersFromConfig created /browse: solr.SearchHandler
   [junit4]   2> 162531 T6296 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 162532 T6296 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 162532 T6296 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.extraction.ExtractingRequestHandler
   [junit4]   2> 162532 T6296 oasc.RequestHandlers.initHandlersFromConfig created /update/extract: solr.extraction.ExtractingRequestHandler
   [junit4]   2> 162532 T6296 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.FieldAnalysisRequestHandler
   [junit4]   2> 162532 T6296 oasc.RequestHandlers.initHandlersFromConfig created /analysis/field: solr.FieldAnalysisRequestHandler
   [junit4]   2> 162532 T6296 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.DocumentAnalysisRequestHandler
   [junit4]   2> 162533 T6296 oasc.RequestHandlers.initHandlersFromConfig created /analysis/document: solr.DocumentAnalysisRequestHandler
   [junit4]   2> 162533 T6296 oasc.RequestHandlers.initHandlersFromConfig created /admin/: solr.admin.AdminHandlers
   [junit4]   2> 162534 T6296 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
   [junit4]   2> 162534 T6296 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
   [junit4]   2> 162534 T6296 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 162534 T6296 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.SearchHandler
   [junit4]   2> 162535 T6296 oasc.RequestHandlers.initHandlersFromConfig created /spell: solr.SearchHandler
   [junit4]   2> 162535 T6296 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.SearchHandler
   [junit4]   2> 162535 T6296 oasc.RequestHandlers.initHandlersFromConfig created /suggest: solr.SearchHandler
   [junit4]   2> 162535 T6296 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.SearchHandler
   [junit4]   2> 162535 T6296 oasc.RequestHandlers.initHandlersFromConfig created /tvrh: solr.SearchHandler
   [junit4]   2> 162535 T6296 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.SearchHandler
   [junit4]   2> 162536 T6296 oasc.RequestHandlers.initHandlersFromConfig created /terms: solr.SearchHandler
   [junit4]   2> 162536 T6296 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.SearchHandler
   [junit4]   2> 162536 T6296 oasc.RequestHandlers.initHandlersFromConfig created /elevate: solr.SearchHandler
   [junit4]   2> 162547 T6296 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 162548 T6296 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 162550 T6296 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 162551 T6296 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 162552 T6296 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 162556 T6296 oasu.CommitTracker.<init> Hard AutoCommit: if uncommited for 15000ms; 
   [junit4]   2> 162556 T6296 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 162557 T6296 oasu.LoggingInfoStream.message [IFD][coreLoadExecutor-200-thread-1]: init: current segments file is "segments_1"; deletionPolicy=org.apache.solr.core.IndexDeletionPolicyWrapper@44dd1ac2
   [junit4]   2> 162558 T6296 oasu.LoggingInfoStream.message [IFD][coreLoadExecutor-200-thread-1]: init: load commit "segments_1"
   [junit4]   2> 162558 T6296 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@1c06dda5 lockFactory=NativeFSLockFactory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/index),segFN=segments_1,generation=1}
   [junit4]   2> 162558 T6296 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 162558 T6296 oasu.LoggingInfoStream.message [IFD][coreLoadExecutor-200-thread-1]: now checkpoint "" [0 segments ; isCommit = false]
   [junit4]   2> 162559 T6296 oasu.LoggingInfoStream.message [IFD][coreLoadExecutor-200-thread-1]: 0 msec to checkpoint
   [junit4]   2> 162559 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: init: create=false
   [junit4]   2> 162559 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: 
   [junit4]   2> 	dir=MockDirectoryWrapper(RAMDirectory@1c06dda5 lockFactory=NativeFSLockFactory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/index)
   [junit4]   2> 	index=
   [junit4]   2> 	version=5.0-SNAPSHOT
   [junit4]   2> 	matchVersion=LUCENE_5_0
   [junit4]   2> 	analyzer=null
   [junit4]   2> 	ramBufferSizeMB=100.0
   [junit4]   2> 	maxBufferedDocs=-1
   [junit4]   2> 	maxBufferedDeleteTerms=-1
   [junit4]   2> 	mergedSegmentWarmer=null
   [junit4]   2> 	delPolicy=org.apache.solr.core.IndexDeletionPolicyWrapper
   [junit4]   2> 	commit=null
   [junit4]   2> 	openMode=APPEND
   [junit4]   2> 	similarity=org.apache.lucene.search.similarities.DefaultSimilarity
   [junit4]   2> 	mergeScheduler=ConcurrentMergeScheduler: maxThreadCount=1, maxMergeCount=2, mergeThreadPriority=-1
   [junit4]   2> 	default WRITE_LOCK_TIMEOUT=1000
   [junit4]   2> 	writeLockTimeout=1000
   [junit4]   2> 	codec=Lucene49
   [junit4]   2> 	infoStream=org.apache.solr.update.LoggingInfoStream
   [junit4]   2> 	mergePolicy=[TieredMergePolicy: maxMergeAtOnce=10, maxMergeAtOnceExplicit=30, maxMergedSegmentMB=5120.0, floorSegmentMB=2.0, forceMergeDeletesPctAllowed=10.0, segmentsPerTier=10.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 	indexerThreadPool=org.apache.lucene.index.DocumentsWriterPerThreadPool@489c9308
   [junit4]   2> 	readerPooling=false
   [junit4]   2> 	perThreadHardLimitMB=1945
   [junit4]   2> 	useCompoundFile=false
   [junit4]   2> 	checkIntegrityAtMerge=false
   [junit4]   2> 	writer=org.apache.lucene.util.SetOnce@1f791776
   [junit4]   2> 	
   [junit4]   2> 162559 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: flush at getReader
   [junit4]   2> 162560 T6296 oasu.LoggingInfoStream.message [DW][coreLoadExecutor-200-thread-1]: startFullFlush
   [junit4]   2> 162560 T6296 oasu.LoggingInfoStream.message [DW][coreLoadExecutor-200-thread-1]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
   [junit4]   2> 162560 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: apply all deletes during flush
   [junit4]   2> 162560 T6296 oasu.LoggingInfoStream.message [BD][coreLoadExecutor-200-thread-1]: prune sis=segments_1:  minGen=9223372036854775807 packetCount=0
   [junit4]   2> 162560 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: return reader version=1 reader=StandardDirectoryReader(segments_1:1:nrt)
   [junit4]   2> 162561 T6296 oasu.LoggingInfoStream.message [DW][coreLoadExecutor-200-thread-1]: coreLoadExecutor-200-thread-1 finishFullFlush success=true
   [junit4]   2> 162561 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: getReader took 2 msec
   [junit4]   2> 162561 T6296 oass.SolrIndexSearcher.<init> Opening Searcher@1117a1f6[collection1] main
   [junit4]   2> 162562 T6296 oasr.ManagedResourceStorage.newStorageIO WARN Cannot write to config directory /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/collection1/conf; switching to use InMemory storage instead.
   [junit4]   2> 162562 T6296 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 162562 T6296 oasr.ManagedResourceStorage.load Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 162562 T6296 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /rest/managed
   [junit4]   2> 162563 T6296 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using InMemoryStorage
   [junit4]   2> 162563 T6296 oasr.RestManager.init Initializing 2 registered ManagedResources
   [junit4]   2> 162563 T6296 oasr.ManagedResourceStorage.load Reading _schema_analysis_stopwords_english.json using InMemoryStorage
   [junit4]   2> 162563 T6296 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /schema/analysis/stopwords/english
   [junit4]   2> 162563 T6296 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _schema_analysis_stopwords_english.json using InMemoryStorage
   [junit4]   2> 162564 T6296 oasrsa.ManagedWordSetResource.onManagedDataLoadedFromStorage Loaded 0 words for /schema/analysis/stopwords/english
   [junit4]   2> 162564 T6296 oasr.ManagedResource.notifyObserversDuringInit Notified 1 observers of /schema/analysis/stopwords/english
   [junit4]   2> 162564 T6296 oasr.ManagedResourceStorage.load Reading _schema_analysis_synonyms_english.json using InMemoryStorage
   [junit4]   2> 162564 T6296 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /schema/analysis/synonyms/english
   [junit4]   2> 162564 T6296 oasrsa.ManagedSynonymFilterFactory$SynonymManager.onManagedDataLoadedFromStorage Loaded 0 synonym mappings for /schema/analysis/synonyms/english
   [junit4]   2> 162565 T6296 oasr.ManagedResource.notifyObserversDuringInit Notified 1 observers of /schema/analysis/synonyms/english
   [junit4]   2> 162565 T6296 oashc.SpellCheckComponent.inform Initializing spell checkers
   [junit4]   2> 162565 T6296 oass.DirectSolrSpellChecker.init init: {name=default,field=text,classname=solr.DirectSolrSpellChecker,distanceMeasure=internal,accuracy=0.5,maxEdits=2,minPrefix=1,maxInspections=5,minQueryLength=4,maxQueryFrequency=0.01}
   [junit4]   2> 162565 T6296 oashc.SpellCheckComponent.inform No queryConverter defined, using default converter
   [junit4]   2> 162566 T6296 oashc.SuggestComponent.inform Initializing SuggestComponent
   [junit4]   2> 162566 T6296 oasss.SolrSuggester.init init: {name=mySuggester,lookupImpl=FuzzyLookupFactory,dictionaryImpl=DocumentDictionaryFactory,field=cat,weightField=price,suggestAnalyzerFieldType=string}
   [junit4]   2> 162566 T6296 oasss.SolrSuggester.init Dictionary loaded with params: {name=mySuggester,lookupImpl=FuzzyLookupFactory,dictionaryImpl=DocumentDictionaryFactory,field=cat,weightField=price,suggestAnalyzerFieldType=string}
   [junit4]   2> 162567 T6296 oashc.QueryElevationComponent.inform Loading QueryElevation from: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/collection1/conf/elevate.xml
   [junit4]   2> 162577 T6296 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 162577 T6297 oasc.QuerySenderListener.newSearcher QuerySenderListener sending requests to Searcher@1117a1f6[collection1] main{UninvertingDirectoryReader()}
   [junit4]   2> 162577 T6296 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 162578 T6286 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1
   [junit4]   2> 162578 T6286 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> ASYNC  NEW_CORE C209 name=collection1 org.apache.solr.core.SolrCore@4fda2aac
   [junit4]   2> 162579 T6297 C209 oasc.SolrCore.execute [collection1] webapp=null path=null params={q=static+firstSearcher+warming+in+solrconfig.xml&distrib=false&event=firstSearcher} hits=0 status=0 QTime=2 
   [junit4]   2> 162579 T6286 oas.SolrJettyTestBase.createJetty Jetty Assigned Port#59691
   [junit4]   2> 162579 T6297 oasc.QuerySenderListener.newSearcher QuerySenderListener done.
   [junit4]   2> 162580 T6297 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: default
   [junit4]   2> 162580 T6297 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: wordbreak
   [junit4]   2> 162580 T6297 oashc.SuggestComponent$SuggesterListener.newSearcher Loading suggester index for: mySuggester
   [junit4]   2> 162581 T6297 oasss.SolrSuggester.reload reload()
   [junit4]   2> 162581 T6297 oasss.SolrSuggester.build build()
   [junit4]   2> 162585 T6297 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1117a1f6[collection1] main{UninvertingDirectoryReader()}
   [junit4]   2> 162590 T6286 oas.SolrTestCaseJ4.setUp ###Starting testChineseDefaults
   [junit4]   2> 162595 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=true hasTickets:false pendingChangesInFullFlush: false
   [junit4]   2> 162596 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: nrtIsCurrent: infoVersion matches: true; DW changes: true; BD changes: false
   [junit4]   2> 162596 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=true hasTickets:false pendingChangesInFullFlush: false
   [junit4]   2> 162597 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: flush at getReader
   [junit4]   2> 162597 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: startFullFlush
   [junit4]   2> 162598 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=true hasTickets:false pendingChangesInFullFlush: false
   [junit4]   2> 162598 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: qtp1189521383-6289: flush naked frozen global deletes
   [junit4]   2> 162599 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: push deletes  1 deleted queries bytesUsed=64 delGen=2 packetCount=1 totBytesUsed=64
   [junit4]   2> 162599 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: flush: push buffered updates:  1 deleted queries bytesUsed=64
   [junit4]   2> 162600 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: apply all deletes during flush
   [junit4]   2> 162600 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: prune sis=segments_1:  minGen=9223372036854775807 packetCount=1
   [junit4]   2> 162600 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: pruneDeletes: prune 1 packets; 0 packets remain
   [junit4]   2> 162601 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: return reader version=1 reader=StandardDirectoryReader(segments_1:1:nrt)
   [junit4]   2> 162601 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: qtp1189521383-6289 finishFullFlush success=true
   [junit4]   2> 162602 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: getReader took 5 msec
   [junit4]   2> 162603 T6289 C209 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2.2&wt=xml} {deleteByQuery=*:* (-1475865354122035200)} 0 10
   [junit4]   2> 162606 T6289 C209 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 162607 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
   [junit4]   2> 162607 T6289 C209 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 162608 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
   [junit4]   2> 162608 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: nrtIsCurrent: infoVersion matches: true; DW changes: false; BD changes: false
   [junit4]   2> 162608 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
   [junit4]   2> 162609 T6289 C209 oasc.SolrCore.openNewSearcher SolrIndexSearcher has not changed - not re-opening: org.apache.solr.search.SolrIndexSearcher
   [junit4]   2> 162609 T6289 C209 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 162610 T6289 C209 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={waitSearcher=true&softCommit=false&version=2.2&wt=xml&commit=true} {commit=} 0 4
   [junit4]   2> 162613 T6289 C209 oasc.SolrCore.execute [collection1] webapp=/solr path=/select params={q=*:*&version=2.2&wt=xml} hits=0 status=0 QTime=0 
   [junit4]   2> 162614 T6286 oascsi.ConcurrentUpdateSolrServer.request blocking for commit/optimize
   [junit4]   2> 162620 T6289 C209 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 162620 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: start
   [junit4]   2> 162621 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: enter lock
   [junit4]   2> 162621 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: now prepare
   [junit4]   2> 162622 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: prepareCommit: flush
   [junit4]   2> 162622 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]:   index before flush 
   [junit4]   2> 162622 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: startFullFlush
   [junit4]   2> 162623 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=1 deletes=true hasTickets:false pendingChangesInFullFlush: false
   [junit4]   2> 162623 T6289 C209 oasu.LoggingInfoStream.message [DWFC][qtp1189521383-6289]: addFlushableState DocumentsWriterPerThread [pendingDeletes=gen=0, segment=_0, aborting=false, numDocsInRAM=1, deleteQueue=DWDQ: [ generation: 2 ]]
   [junit4]   2> 162624 T6289 C209 oasu.LoggingInfoStream.message [DWPT][qtp1189521383-6289]: flush postings as segment _0 numDocs=1
   [junit4]   2> 162627 T6289 C209 oasu.LoggingInfoStream.message [DWPT][qtp1189521383-6289]: new segment has 0 deleted docs
   [junit4]   2> 162628 T6289 C209 oasu.LoggingInfoStream.message [DWPT][qtp1189521383-6289]: new segment has no vectors; norms; no docValues; prox; freqs
   [junit4]   2> 162628 T6289 C209 oasu.LoggingInfoStream.message [DWPT][qtp1189521383-6289]: flushedFiles=[_0_Lucene41_0.tip, _0_Lucene41_0.tim, _0.fnm, _0.nvm, _0.fdx, _0.nvd, _0.fdt, _0_Lucene41_0.pos, _0_Lucene41_0.doc]
   [junit4]   2> 162629 T6289 C209 oasu.LoggingInfoStream.message [DWPT][qtp1189521383-6289]: flushed codec=Lucene49
   [junit4]   2> 162629 T6289 C209 oasu.LoggingInfoStream.message [DWPT][qtp1189521383-6289]: flushed: segment=_0 ramUsed=0.071 MB newFlushedSize(includes docstores)=0.001 MB docs/MB=979.063
   [junit4]   2> 162630 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: publishFlushedSegment seg-private updates=null
   [junit4]   2> 162631 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: publishFlushedSegment
   [junit4]   2> 162631 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: push deletes  1 deleted terms (unique count=1) bytesUsed=1056 delGen=4 packetCount=1 totBytesUsed=1056
   [junit4]   2> 162631 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: publish sets newSegment delGen=5 seg=_0(5.0):C1
   [junit4]   2> 162632 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: now checkpoint "_0(5.0):C1" [1 segments ; isCommit = false]
   [junit4]   2> 162632 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: 0 msec to checkpoint
   [junit4]   2> 162633 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: apply all deletes during flush
   [junit4]   2> 162633 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: applyDeletes: infos=[_0(5.0):C1] packetCount=1
   [junit4]   2> 162634 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: applyDeletes took 1 msec
   [junit4]   2> 162634 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: prune sis=segments_1: _0(5.0):C1 minGen=6 packetCount=1
   [junit4]   2> 162634 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: pruneDeletes: prune 1 packets; 0 packets remain
   [junit4]   2> 162635 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: qtp1189521383-6289 finishFullFlush success=true
   [junit4]   2> 162635 T6289 C209 oasu.LoggingInfoStream.message [TMP][qtp1189521383-6289]: findMerges: 1 segments
   [junit4]   2> 162636 T6289 C209 oasu.LoggingInfoStream.message [TMP][qtp1189521383-6289]:   seg=_0(5.0):C1 size=0.001 MB [floored]
   [junit4]   2> 162636 T6289 C209 oasu.LoggingInfoStream.message [TMP][qtp1189521383-6289]:   allowedSegmentCount=1 vs count=1 (eligible count=1) tooBigCount=0
   [junit4]   2> 162637 T6289 C209 oasu.LoggingInfoStream.message [CMS][qtp1189521383-6289]: now merge
   [junit4]   2> 162637 T6289 C209 oasu.LoggingInfoStream.message [CMS][qtp1189521383-6289]:   index: _0(5.0):C1
   [junit4]   2> 162638 T6289 C209 oasu.LoggingInfoStream.message [CMS][qtp1189521383-6289]:   no more merges pending; now return
   [junit4]   2> 162638 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: startCommit(): start
   [junit4]   2> 162638 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: startCommit index=_0(5.0):C1 changeCount=3
   [junit4]   2> 162639 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: done all syncs: [_0_Lucene41_0.tip, _0.si, _0_Lucene41_0.tim, _0.nvm, _0.fnm, _0.fdx, _0.nvd, _0.fdt, _0_Lucene41_0.doc, _0_Lucene41_0.pos]
   [junit4]   2> 162640 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: pendingCommit != null
   [junit4]   2> 162640 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: wrote segments file "segments_2"
   [junit4]   2> 162641 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: now checkpoint "_0(5.0):C1" [1 segments ; isCommit = true]
   [junit4]   2> 162641 T6289 C209 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@1c06dda5 lockFactory=NativeFSLockFactory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/index),segFN=segments_1,generation=1}
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@1c06dda5 lockFactory=NativeFSLockFactory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/index),segFN=segments_2,generation=2}
   [junit4]   2> 162642 T6289 C209 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
   [junit4]   2> 162642 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: deleteCommits: now decRef commit "segments_1"
   [junit4]   2> 162643 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: delete "segments_1"
   [junit4]   2> 162643 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: 2 msec to checkpoint
   [junit4]   2> 162644 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: took 22.2 msec
   [junit4]   2> 162644 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: done
   [junit4]   2> 162645 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
   [junit4]   2> 162645 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: nrtIsCurrent: infoVersion matches: false; DW changes: false; BD changes: false
   [junit4]   2> 162646 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: flush at getReader
   [junit4]   2> 162646 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: startFullFlush
   [junit4]   2> 162646 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
   [junit4]   2> 162647 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: apply all deletes during flush
   [junit4]   2> 162647 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: applyDeletes: no deletes; skipping
   [junit4]   2> 162648 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: prune sis=segments_2: _0(5.0):C1 minGen=6 packetCount=0
   [junit4]   2> 162650 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: return reader version=3 reader=StandardDirectoryReader(segments_2:3:nrt _0(5.0):C1)
   [junit4]   2> 162650 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: qtp1189521383-6289 finishFullFlush success=true
   [junit4]   2> 162650 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: getReader took 4 msec
   [junit4]   2> 162651 T6289 C209 oass.SolrIndexSearcher.<init> Opening Searcher@755b797[collection1] main
   [junit4]   2> 162652 T6289 C209 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 162652 T6297 oasc.QuerySenderListener.newSearcher QuerySenderListener sending requests to Searcher@755b797[collection1] main{UninvertingDirectoryReader(Uninverting(_0(5.0):C1))}
   [junit4]   2> 162653 T6297 oasc.QuerySenderListener.newSearcher QuerySenderListener done.
   [junit4]   2> 162653 T6297 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@755b797[collection1] main{UninvertingDirectoryReader(Uninverting(_0(5.0):C1))}
   [junit4]   2> 162653 T6289 C209 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={waitSearcher=true&softCommit=false&version=2.2&wt=xml&commit=true} {add=[42 (1475865354146152448)],commit=} 0 36
   [junit4]   2> 162659 T6289 C209 oasc.SolrCore.execute [collection1] webapp=/solr path=/select params={q=????&version=2.2&wt=xml} hits=1 status=0 QTime=2 
   [junit4]   2> 162661 T6286 oas.SolrTestCaseJ4.tearDown ###Ending testChineseDefaults
   [junit4]   2> 162674 T6286 oas.SolrTestCaseJ4.setUp ###Starting testUpdateField
   [junit4]   2> 162679 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=true hasTickets:false pendingChangesInFullFlush: false
   [junit4]   2> 162679 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: nrtIsCurrent: infoVersion matches: true; DW changes: true; BD changes: false
   [junit4]   2> 162680 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=true hasTickets:false pendingChangesInFullFlush: false
   [junit4]   2> 162680 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: flush at getReader
   [junit4]   2> 162681 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: startFullFlush
   [junit4]   2> 162681 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=true hasTickets:false pendingChangesInFullFlush: false
   [junit4]   2> 162681 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: qtp1189521383-6289: flush naked frozen global deletes
   [junit4]   2> 162682 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: push deletes  1 deleted queries bytesUsed=64 delGen=8 packetCount=1 totBytesUsed=64
   [junit4]   2> 162682 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: flush: push buffered updates:  1 deleted queries bytesUsed=64
   [junit4]   2> 162683 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: apply all deletes during flush
   [junit4]   2> 162683 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: applyDeletes: infos=[_0(5.0):C1] packetCount=1
   [junit4]   2> 162854 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: seg=_0(5.0):C1 segGen=6 coalesced deletes=[CoalescedUpdates(termSets=1,queries=1,numericDVUpdates=0,binaryDVUpdates=0)] newDelCount=1 100% deleted
   [junit4]   2> 162855 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: applyDeletes took 172 msec
   [junit4]   2> 162856 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: now checkpoint "_0(5.0):C1/1" [1 segments ; isCommit = false]
   [junit4]   2> 162856 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: 0 msec to checkpoint
   [junit4]   2> 162857 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: drop 100% deleted segments: _0(5.0):C1/1
   [junit4]   2> 162857 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: now checkpoint "" [0 segments ; isCommit = false]
   [junit4]   2> 162857 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: 0 msec to checkpoint
   [junit4]   2> 162858 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: prune sis=segments_2:  minGen=9223372036854775807 packetCount=1
   [junit4]   2> 162858 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: pruneDeletes: prune 1 packets; 0 packets remain
   [junit4]   2> 162859 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: return reader version=5 reader=StandardDirectoryReader(segments_2:5:nrt)
   [junit4]   2> 162859 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: qtp1189521383-6289 finishFullFlush success=true
   [junit4]   2> 162860 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: getReader took 180 msec
   [junit4]   2> 162860 T6289 C209 oass.SolrIndexSearcher.<init> Opening Searcher@1119083d[collection1] realtime
   [junit4]   2> 162861 T6289 C209 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2.2&wt=xml} {deleteByQuery=*:* (-1475865354209067008)} 0 184
   [junit4]   2> 162865 T6289 C209 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 162866 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: start
   [junit4]   2> 162866 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: enter lock
   [junit4]   2> 162866 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: now prepare
   [junit4]   2> 162867 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: prepareCommit: flush
   [junit4]   2> 162867 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]:   index before flush 
   [junit4]   2> 162868 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: startFullFlush
   [junit4]   2> 162868 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
   [junit4]   2> 162868 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: apply all deletes during flush
   [junit4]   2> 162869 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: prune sis=segments_2:  minGen=9223372036854775807 packetCount=0
   [junit4]   2> 162869 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: qtp1189521383-6289 finishFullFlush success=true
   [junit4]   2> 162870 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: startCommit(): start
   [junit4]   2> 162870 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: startCommit index= changeCount=6
   [junit4]   2> 162871 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: done all syncs: []
   [junit4]   2> 162871 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: pendingCommit != null
   [junit4]   2> 162872 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: wrote segments file "segments_3"
   [junit4]   2> 162872 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: now checkpoint "" [0 segments ; isCommit = true]
   [junit4]   2> 162873 T6289 C209 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@1c06dda5 lockFactory=NativeFSLockFactory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/index),segFN=segments_2,generation=2}
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@1c06dda5 lockFactory=NativeFSLockFactory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/index),segFN=segments_3,generation=3}
   [junit4]   2> 162873 T6289 C209 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
   [junit4]   2> 162874 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: deleteCommits: now decRef commit "segments_2"
   [junit4]   2> 162874 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: delete "segments_2"
   [junit4]   2> 162875 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: 2 msec to checkpoint
   [junit4]   2> 162875 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: took 8.1 msec
   [junit4]   2> 162876 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: done
   [junit4]   2> 162876 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
   [junit4]   2> 162876 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: nrtIsCurrent: infoVersion matches: true; DW changes: false; BD changes: false
   [junit4]   2> 162877 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
   [junit4]   2> 162877 T6289 C209 oass.SolrIndexSearcher.<init> Opening Searcher@2776bcc3[collection1] main
   [junit4]   2> 162878 T6289 C209 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 162878 T6297 oasc.QuerySenderListener.newSearcher QuerySenderListener sending requests to Searcher@2776bcc3[collection1] main{UninvertingDirectoryReader()}
   [junit4]   2> 162879 T6297 oasc.QuerySenderListener.newSearcher QuerySenderListener done.
   [junit4]   2> 162879 T6297 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2776bcc3[collection1] main{UninvertingDirectoryReader()}
   [junit4]   2> 162880 T6297 oasu.LoggingInfoStream.message [IFD][searcherExecutor-201-thread-1]: delete "_0_Lucene41_0.tip"
   [junit4]   2> 162880 T6297 oasu.LoggingInfoStream.message [IFD][searcherExecutor-201-thread-1]: delete "_0.si"
   [junit4]   2> 162880 T6297 oasu.LoggingInfoStream.message [IFD][searcherExecutor-201-thread-1]: delete "_0_Lucene41_0.tim"
   [junit4]   2> 162881 T6297 oasu.LoggingInfoStream.message [IFD][searcherExecutor-201-thread-1]: delete "_0.nvm"
   [junit4]   2> 162881 T6297 oasu.LoggingInfoStream.message [IFD][searcherExecutor-201-thread-1]: delete "_0.fnm"
   [junit4]   2> 162881 T6297 oasu.LoggingInfoStream.message [IFD][searcherExecutor-201-thread-1]: delete "_0.fdx"
   [junit4]   2> 162881 T6297 oasu.LoggingInfoStream.message [IFD][searcherExecutor-201-thread-1]: delete "_0.nvd"
   [junit4]   2> 162882 T6297 oasu.LoggingInfoStream.message [IFD][searcherExecutor-201-thread-1]: delete "_0.fdt"
   [junit4]   2> 162882 T6297 oasu.LoggingInfoStream.message [IFD][searcherExecutor-201-thread-1]: delete "_0_Lucene41_0.doc"
   [junit4]   2> 162882 T6297 oasu.LoggingInfoStream.message [IFD][searcherExecutor-201-thread-1]: delete "_0_Lucene41_0.pos"
   [junit4]   2> 162882 T6289 C209 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={waitSearcher=true&softCommit=false&version=2.2&wt=xml&commit=true} {commit=} 0 17
   [junit4]   2> 163137 T6289 C209 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2.2&wt=xml} {add=[unique (1475865354429267968)]} 0 250
   [junit4]   2> 163140 T6289 C209 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 163140 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: start
   [junit4]   2> 163141 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: enter lock
   [junit4]   2> 163141 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: now prepare
   [junit4]   2> 163141 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: prepareCommit: flush
   [junit4]   2> 163141 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]:   index before flush 
   [junit4]   2> 163142 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: startFullFlush
   [junit4]   2> 163142 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=1 deletes=true hasTickets:false pendingChangesInFullFlush: false
   [junit4]   2> 163142 T6289 C209 oasu.LoggingInfoStream.message [DWFC][qtp1189521383-6289]: addFlushableState DocumentsWriterPerThread [pendingDeletes=gen=0, segment=_1, aborting=false, numDocsInRAM=1, deleteQueue=DWDQ: [ generation: 6 ]]
   [junit4]   2> 163143 T6289 C209 oasu.LoggingInfoStream.message [DWPT][qtp1189521383-6289]: flush postings as segment _1 numDocs=1
   [junit4]   2> 163145 T6289 C209 oasu.LoggingInfoStream.message [DWPT][qtp1189521383-6289]: new segment has 0 deleted docs
   [junit4]   2> 163145 T6289 C209 oasu.LoggingInfoStream.message [DWPT][qtp1189521383-6289]: new segment has no vectors; norms; no docValues; prox; freqs
   [junit4]   2> 163145 T6289 C209 oasu.LoggingInfoStream.message [DWPT][qtp1189521383-6289]: flushedFiles=[_1.nvd, _1_Lucene41_0.doc, _1_Lucene41_0.tip, _1_Lucene41_0.tim, _1.nvm, _1.fdt, _1.fdx, _1.fnm, _1_Lucene41_0.pos]
   [junit4]   2> 163146 T6289 C209 oasu.LoggingInfoStream.message [DWPT][qtp1189521383-6289]: flushed codec=Lucene49
   [junit4]   2> 163146 T6289 C209 oasu.LoggingInfoStream.message [DWPT][qtp1189521383-6289]: flushed: segment=_1 ramUsed=0.079 MB newFlushedSize(includes docstores)=0.001 MB docs/MB=757.642
   [junit4]   2> 163146 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: publishFlushedSegment seg-private updates=null
   [junit4]   2> 163147 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: publishFlushedSegment
   [junit4]   2> 163147 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: push deletes  1 deleted terms (unique count=1) bytesUsed=1056 delGen=11 packetCount=1 totBytesUsed=1056
   [junit4]   2> 163147 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: publish sets newSegment delGen=12 seg=_1(5.0):C1
   [junit4]   2> 163148 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: now checkpoint "_1(5.0):C1" [1 segments ; isCommit = false]
   [junit4]   2> 163148 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: 0 msec to checkpoint
   [junit4]   2> 163148 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: apply all deletes during flush
   [junit4]   2> 163149 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: applyDeletes: infos=[_1(5.0):C1] packetCount=1
   [junit4]   2> 163149 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: applyDeletes took 0 msec
   [junit4]   2> 163149 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: prune sis=segments_3: _1(5.0):C1 minGen=13 packetCount=1
   [junit4]   2> 163149 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: pruneDeletes: prune 1 packets; 0 packets remain
   [junit4]   2> 163150 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: qtp1189521383-6289 finishFullFlush success=true
   [junit4]   2> 163150 T6289 C209 oasu.LoggingInfoStream.message [TMP][qtp1189521383-6289]: findMerges: 1 segments
   [junit4]   2> 163150 T6289 C209 oasu.LoggingInfoStream.message [TMP][qtp1189521383-6289]:   seg=_1(5.0):C1 size=0.001 MB [floored]
   [junit4]   2> 163151 T6289 C209 oasu.LoggingInfoStream.message [TMP][qtp1189521383-6289]:   allowedSegmentCount=1 vs count=1 (eligible count=1) tooBigCount=0
   [junit4]   2> 163151 T6289 C209 oasu.LoggingInfoStream.message [CMS][qtp1189521383-6289]: now merge
   [junit4]   2> 163151 T6289 C209 oasu.LoggingInfoStream.message [CMS][qtp1189521383-6289]:   index: _1(5.0):C1
   [junit4]   2> 163151 T6289 C209 oasu.LoggingInfoStream.message [CMS][qtp1189521383-6289]:   no more merges pending; now return
   [junit4]   2> 163152 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: startCommit(): start
   [junit4]   2> 163152 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: startCommit index=_1(5.0):C1 changeCount=9
   [junit4]   2> 163153 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: done all syncs: [_1.nvd, _1_Lucene41_0.doc, _1_Lucene41_0.tip, _1.fdt, _1.nvm, _1_Lucene41_0.tim, _1.si, _1.fdx, _1_Lucene41_0.pos, _1.fnm]
   [junit4]   2> 163153 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: pendingCommit != null
   [junit4]   2> 163153 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: wrote segments file "segments_4"
   [junit4]   2> 163153 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: now checkpoint "_1(5.0):C1" [1 segments ; isCommit = true]
   [junit4]   2> 163154 T6289 C209 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@1c06dda5 lockFactory=NativeFSLockFactory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/index),segFN=segments_3,generation=3}
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@1c06dda5 lockFactory=NativeFSLockFactory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/index),segFN=segments_4,generation=4}
   [junit4]   2> 163154 T6289 C209 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
   [junit4]   2> 163154 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: deleteCommits: now decRef commit "segments_3"
   [junit4]   2> 163155 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: delete "segments_3"
   [junit4]   2> 163155 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: 1 msec to checkpoint
   [junit4]   2> 163155 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: took 14.2 msec
   [junit4]   2> 163156 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: done
   [junit4]   2> 163156 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
   [junit4]   2> 163156 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: nrtIsCurrent: infoVersion matches: false; DW changes: false; BD changes: false
   [junit4]   2> 163157 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: flush at getReader
   [junit4]   2> 163157 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: startFullFlush
   [junit4]   2> 163157 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
   [junit4]   2> 163157 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: apply all deletes during flush
   [junit4]   2> 163158 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: applyDeletes: no deletes; skipping
   [junit4]   2> 163158 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: prune sis=segments_4: _1(5.0):C1 minGen=13 packetCount=0
   [junit4]   2> 163159 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: return reader version=7 reader=StandardDirectoryReader(segments_4:7:nrt _1(5.0):C1)
   [junit4]   2> 163160 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: qtp1189521383-6289 finishFullFlush success=true
   [junit4]   2> 163160 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: getReader took 3 msec
   [junit4]   2> 163160 T6289 C209 oass.SolrIndexSearcher.<init> Opening Searcher@76e4acbe[collection1] main
   [junit4]   2> 163161 T6289 C209 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 163161 T6297 oasc.QuerySenderListener.newSearcher QuerySenderListener sending requests to Searcher@76e4acbe[collection1] main{UninvertingDirectoryReader(Uninverting(_1(5.0):C1))}
   [junit4]   2> 163161 T6297 oasc.QuerySenderListener.newSearcher QuerySenderListener done.
   [junit4]   2> 163162 T6297 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@76e4acbe[collection1] main{UninvertingDirectoryReader(Uninverting(_1(5.0):C1))}
   [junit4]   2> 163162 T6289 C209 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={waitSearcher=true&softCommit=false&version=2.2&wt=xml&commit=true} {commit=} 0 22
   [junit4]   2> 163173 T6289 C209 oasc.SolrCore.execute [collection1] webapp=/solr path=/select params={q=*:*&fl=id,price_f,name,_version_&version=2.2&wt=xml} hits=1 status=0 QTime=8 
   [junit4]   2> 163179 T6289 C209 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2.2&wt=xml} {} 0 2
   [junit4]   2> 163430 T6289 C209 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 163431 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
   [junit4]   2> 163431 T6289 C209 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 163431 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
   [junit4]   2> 163432 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: nrtIsCurrent: infoVersion matches: true; DW changes: false; BD changes: false
   [junit4]   2> 163432 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
   [junit4]   2> 163432 T6289 C209 oasc.SolrCore.openNewSearcher SolrIndexSearcher has not changed - not re-opening: org.apache.solr.search.SolrIndexSearcher
   [junit4]   2> 163433 T6289 C209 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 163433 T6289 C209 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={waitSearcher=true&softCommit=false&version=2.2&wt=xml&commit=true} {commit=} 0 3
   [junit4]   2> 163688 T6289 C209 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2.2&wt=xml} {add=[unique (1475865355007033344)]} 0 251
   [junit4]   2> 163692 T6289 C209 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 163693 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: start
   [jun

[...truncated too long message...]

  2> 183797 T6289 C211 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
   [junit4]   2> 183797 T6289 C211 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: apply all deletes during flush
   [junit4]   2> 183798 T6289 C211 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: applyDeletes: no deletes; skipping
   [junit4]   2> 183798 T6289 C211 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: prune sis=segments_20: _25(5.0):C2 _22(5.0):C2 _26(5.0):C1 _23(5.0):C3 _24(5.0):C2 minGen=338 packetCount=0
   [junit4]   2> 183806 T6289 C211 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: return reader version=239 reader=StandardDirectoryReader(segments_20:239:nrt _25(5.0):C2 _22(5.0):C2 _26(5.0):C1 _23(5.0):C3 _24(5.0):C2)
   [junit4]   2> 183806 T6289 C211 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: qtp1189521383-6289 finishFullFlush success=true
   [junit4]   2> 183807 T6289 C211 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: getReader took 11 msec
   [junit4]   2> 183808 T6289 C211 oass.SolrIndexSearcher.<init> Opening Searcher@132c9694[collection1] main
   [junit4]   2> 183808 T6297 oasc.QuerySenderListener.newSearcher QuerySenderListener sending requests to Searcher@132c9694[collection1] main{UninvertingDirectoryReader(Uninverting(_25(5.0):C2) Uninverting(_22(5.0):C2) Uninverting(_26(5.0):C1) Uninverting(_23(5.0):C3) Uninverting(_24(5.0):C2))}
   [junit4]   2> 183809 T6297 oasc.QuerySenderListener.newSearcher QuerySenderListener done.
   [junit4]   2> 183809 T6297 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@132c9694[collection1] main{UninvertingDirectoryReader(Uninverting(_25(5.0):C2) Uninverting(_22(5.0):C2) Uninverting(_26(5.0):C1) Uninverting(_23(5.0):C3) Uninverting(_24(5.0):C2))}
   [junit4]   2> 183809 T6289 C211 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 183810 T6289 C211 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={waitSearcher=true&softCommit=false&version=2.2&wt=xml&commit=true} {commit=} 0 68
   [junit4]   2> 183814 T6289 C211 oasc.SolrCore.execute [collection1] webapp=/solr path=/select params={q=inStock:true&version=2.2&wt=xml} hits=5 status=0 QTime=1 
   [junit4]   2> 183819 T6289 C211 oasc.SolrCore.execute [collection1] webapp=/solr path=/select params={q=inStock:false&version=2.2&wt=xml} hits=5 status=0 QTime=1 
   [junit4]   2> 183861 T6289 C211 oasc.SolrCore.execute [collection1] webapp=/solr path=/select params={stats.field=val_i&rows=0&q=*:*&version=2.2&wt=xml&stats=true&f.val_i.stats.facet=inStock} hits=10 status=0 QTime=38 
   [junit4]   2> 183863 T6286 oas.SolrTestCaseJ4.tearDown ###Ending testStatistics
   [junit4]   2> 183866 T6286 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=761401314
   [junit4]   2> 183867 T6286 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@4fda2aac
   [junit4]   2> 183871 T6286 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=75,autocommit maxTime=15000ms,autocommits=0,soft autocommits=3,optimizes=8,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=1,cumulative_adds=359,cumulative_deletesById=5,cumulative_deletesByQuery=34,cumulative_errors=1,transaction_logs_total_size=3981,transaction_logs_total_number=10}
   [junit4]   2> 183871 T6286 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 183872 T6286 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 183872 T6286 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 183873 T6286 C211 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleStreamingTest-seed#[25EEF9A8D20C97FB]-worker]: waitForMerges
   [junit4]   2> 183873 T6286 C211 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleStreamingTest-seed#[25EEF9A8D20C97FB]-worker]: waitForMerges done
   [junit4]   2> 183874 T6286 C211 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleStreamingTest-seed#[25EEF9A8D20C97FB]-worker]: rollback
   [junit4]   2> 183874 T6286 C211 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleStreamingTest-seed#[25EEF9A8D20C97FB]-worker]: all running merges have aborted
   [junit4]   2> 183874 T6286 C211 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleStreamingTest-seed#[25EEF9A8D20C97FB]-worker]: rollback: done finish merges
   [junit4]   2> 183875 T6286 C211 oasu.LoggingInfoStream.message [DW][SUITE-SolrExampleStreamingTest-seed#[25EEF9A8D20C97FB]-worker]: abort
   [junit4]   2> 183875 T6286 C211 oasu.LoggingInfoStream.message [DW][SUITE-SolrExampleStreamingTest-seed#[25EEF9A8D20C97FB]-worker]: done abort; abortedFiles=[] success=true
   [junit4]   2> 183876 T6286 C211 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleStreamingTest-seed#[25EEF9A8D20C97FB]-worker]: rollback: infos=_25(5.0):C2 _22(5.0):C2 _26(5.0):C1 _23(5.0):C3 _24(5.0):C2
   [junit4]   2> 183876 T6286 C211 oasu.LoggingInfoStream.message [IFD][SUITE-SolrExampleStreamingTest-seed#[25EEF9A8D20C97FB]-worker]: now checkpoint "_25(5.0):C2 _22(5.0):C2 _26(5.0):C1 _23(5.0):C3 _24(5.0):C2" [5 segments ; isCommit = false]
   [junit4]   2> 183877 T6286 C211 oasu.LoggingInfoStream.message [IFD][SUITE-SolrExampleStreamingTest-seed#[25EEF9A8D20C97FB]-worker]: 0 msec to checkpoint
   [junit4]   2> 183878 T6286 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 183879 T6286 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 183880 T6286 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001 [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001;done=false>>]
   [junit4]   2> 183880 T6286 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001
   [junit4]   2> 183880 T6286 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/index [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/index;done=false>>]
   [junit4]   2> 183881 T6286 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/index
   [junit4]   2> 183881 T6286 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
   [junit4]   2> 188870 T6286 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/./temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001
   [junit4]   2> NOTE: test params are: codec=Lucene40, sim=DefaultSimilarity, locale=tr_TR, timezone=America/Buenos_Aires
   [junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_60 (64-bit)/cpus=16,threads=1,free=83553456,total=268435456
   [junit4]   2> NOTE: All tests run in this JVM: [ExternalHttpClientTest, MergeIndexesEmbeddedTest, TestSolrProperties, TestLBHttpSolrServer, BasicHttpSolrServerTest, FieldAnalysisResponseTest, QueryResponseTest, ConcurrentUpdateSolrServerTest, NoOpResponseParserTest, SolrExampleStreamingTest]
   [junit4] Completed on J1 in 27.50s, 28 tests, 1 error <<< FAILURES!

[...truncated 79 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:474: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:454: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:45: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:202: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:496: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1298: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:922: There were test failures: 51 suites, 299 tests, 1 error

Total time: 112 minutes 13 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-Tests-trunk-Java7 #4781
Archived 3 artifacts
Archive block size is 32768
Received 0 blocks and 10361750 bytes
Compression is 0.0%
Took 3.1 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4791 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4791/

1 tests failed.
FAILED:  org.apache.solr.cloud.MultiThreadedOCPTest.testDistribSearch

Error Message:
We have a failed SPLITSHARD task

Stack Trace:
java.lang.AssertionError: We have a failed SPLITSHARD task
	at __randomizedtesting.SeedInfo.seed([9BBB3C28E81C7420:1A5DB2309F43141C]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.MultiThreadedOCPTest.testTaskExclusivity(MultiThreadedOCPTest.java:125)
	at org.apache.solr.cloud.MultiThreadedOCPTest.doTest(MultiThreadedOCPTest.java:71)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:865)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 13795 lines...]
   [junit4] Suite: org.apache.solr.cloud.MultiThreadedOCPTest
   [junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/init-core-data-001
   [junit4]   2> 3161990 T8786 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 3161990 T8786 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /_x/c
   [junit4]   2> 3161996 T8786 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> 3161997 T8786 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 3161997 T8787 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 3162097 T8786 oasc.ZkTestServer.run start zk server on port:37683
   [junit4]   2> 3162098 T8786 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3162103 T8793 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@24f55b5f name:ZooKeeperConnection Watcher:127.0.0.1:37683 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3162103 T8786 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3162104 T8786 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 3162107 T8786 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3162108 T8795 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f20083b name:ZooKeeperConnection Watcher:127.0.0.1:37683/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3162108 T8786 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3162109 T8786 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 3162111 T8786 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 3162113 T8786 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 3162114 T8786 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 3162116 T8786 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 3162117 T8786 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 3162119 T8786 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 3162120 T8786 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 3162223 T8786 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3162223 T8786 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3162226 T8786 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 3162226 T8786 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 3162228 T8786 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 3162229 T8786 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 3162231 T8786 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 3162232 T8786 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 3162234 T8786 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 3162234 T8786 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 3162236 T8786 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3162237 T8786 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3162239 T8786 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3162239 T8786 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3162241 T8786 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 3162242 T8786 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 3162244 T8786 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 3162245 T8786 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 3163457 T8786 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 3163462 T8786 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:65517
   [junit4]   2> 3163463 T8786 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 3163463 T8786 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3163463 T8786 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-002
   [junit4]   2> 3163464 T8786 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-002/'
   [junit4]   2> 3163518 T8786 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-002/solr.xml
   [junit4]   2> 3163600 T8786 oasc.CoreContainer.<init> New CoreContainer 1435050643
   [junit4]   2> 3163601 T8786 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-002/]
   [junit4]   2> 3163602 T8786 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 3163603 T8786 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 3163603 T8786 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 3163604 T8786 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 3163604 T8786 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 3163604 T8786 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 3163605 T8786 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 3163605 T8786 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 3163606 T8786 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 3163606 T8786 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 3163607 T8786 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3163607 T8786 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3163608 T8786 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 3163608 T8786 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:37683/solr
   [junit4]   2> 3163608 T8786 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 3163609 T8786 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3163611 T8806 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e3d0cc4 name:ZooKeeperConnection Watcher:127.0.0.1:37683 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3163611 T8786 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3163613 T8786 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3163614 T8808 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5c44a9cd name:ZooKeeperConnection Watcher:127.0.0.1:37683/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3163615 T8786 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3163616 T8786 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 3163618 T8786 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 3163620 T8786 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 3163622 T8786 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 3163624 T8786 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 3163626 T8786 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 3163627 T8786 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:65517__x%2Fc
   [junit4]   2> 3163628 T8786 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:65517__x%2Fc
   [junit4]   2> 3163630 T8786 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 3163631 T8786 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 3163634 T8786 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:65517__x%2Fc
   [junit4]   2> 3163634 T8786 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 3163635 T8786 oasc.Overseer.start Overseer (id=92236290143027203-127.0.0.1:65517__x%2Fc-n_0000000000) starting
   [junit4]   2> 3163637 T8786 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 3163643 T8810 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 3163643 T8786 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 3163645 T8786 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 3163646 T8786 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3163648 T8809 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 3163652 T8811 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
   [junit4]   2> 3163652 T8811 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 3163653 T8811 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 3163653 T8812 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3163654 T8809 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 3163655 T8809 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:65517/_x/c",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:65517__x%2Fc",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 3163655 T8809 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
   [junit4]   2> 3163655 T8809 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 3163657 T8812 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3163657 T8812 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3164653 T8811 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 3164654 T8811 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
   [junit4]   2> 3164655 T8811 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 3164655 T8811 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 3164656 T8811 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3164656 T8811 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-002/collection1/'
   [junit4]   2> 3164658 T8811 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-002/collection1/lib/README' to classloader
   [junit4]   2> 3164658 T8811 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-002/collection1/lib/.svn/' to classloader
   [junit4]   2> 3164659 T8811 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-002/collection1/lib/classes/' to classloader
   [junit4]   2> 3164733 T8811 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
   [junit4]   2> 3164784 T8811 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 3164885 T8811 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 3164898 T8811 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 3165306 T8811 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 3165319 T8811 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 3165322 T8811 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 3165334 T8811 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 3165338 T8811 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 3165342 T8811 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3165343 T8811 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 3165343 T8811 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 3165344 T8811 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3165345 T8811 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 3165345 T8811 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 3165345 T8811 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 3165345 T8811 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 3165346 T8811 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-002/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-001/control/data/
   [junit4]   2> 3165346 T8811 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5b4fab9a
   [junit4]   2> 3165347 T8811 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-001/control/data
   [junit4]   2> 3165347 T8811 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-001/control/data/index/
   [junit4]   2> 3165347 T8811 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
   [junit4]   2> 3165348 T8811 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-001/control/data/index
   [junit4]   2> 3165348 T8811 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=31, maxMergeAtOnceExplicit=15, maxMergedSegmentMB=46.5205078125, floorSegmentMB=0.3212890625, forceMergeDeletesPctAllowed=16.008820695250826, segmentsPerTier=46.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.2865870921699244
   [junit4]   2> 3165349 T8811 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@42cdcd4b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@79789efc),segFN=segments_1,generation=1}
   [junit4]   2> 3165350 T8811 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 3165354 T8811 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 3165355 T8811 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 3165355 T8811 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 3165356 T8811 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3165356 T8811 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3165356 T8811 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 3165357 T8811 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3165357 T8811 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3165358 T8811 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 3165358 T8811 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 3165359 T8811 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 3165359 T8811 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 3165360 T8811 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 3165360 T8811 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 3165361 T8811 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 3165361 T8811 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 3165361 T8811 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 3165362 T8811 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 3165362 T8811 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 3165363 T8811 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 3165363 T8811 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 3165364 T8811 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 3165382 T8811 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 3165384 T8811 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 3165386 T8811 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 3165389 T8811 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 3165391 T8811 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 3165397 T8811 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 3165397 T8811 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 3165398 T8811 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=49, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=49.939453125, floorSegmentMB=1.9423828125, forceMergeDeletesPctAllowed=2.637825278493197, segmentsPerTier=45.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 3165399 T8811 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@42cdcd4b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@79789efc),segFN=segments_1,generation=1}
   [junit4]   2> 3165399 T8811 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 3165399 T8811 oass.SolrIndexSearcher.<init> Opening Searcher@2d88cd47[collection1] main
   [junit4]   2> 3165400 T8811 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 3165401 T8811 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3165401 T8811 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 3165401 T8811 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 3165402 T8811 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 3165402 T8811 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3165402 T8811 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 3165403 T8811 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3165403 T8811 oascc.SolrZkClient.makePath makePath: /configs/conf1/_rest_managed.json
   [junit4]   2> 3165405 T8811 oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new znode /configs/conf1/_rest_managed.json
   [junit4]   2> 3165405 T8811 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3165406 T8811 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 3165408 T8813 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2d88cd47[collection1] main{UninvertingDirectoryReader()}
   [junit4]   2> 3165412 T8811 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 3165412 T8816 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:65517/_x/c collection:control_collection shard:shard1
   [junit4]   2> 3165413 T8786 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
   [junit4]   2> 3165413 T8786 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 3165413 T8816 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 3165415 T8786 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3165416 T8818 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@23e0a6d1 name:ZooKeeperConnection Watcher:127.0.0.1:37683/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3165416 T8786 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3165419 T8786 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3165420 T8816 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 3165421 T8786 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 3165422 T8812 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3165422 T8816 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 3165422 T8816 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C1447 name=collection1 org.apache.solr.core.SolrCore@1c4d38c1 url=http://127.0.0.1:65517/_x/c/collection1 node=127.0.0.1:65517__x%2Fc C1447_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:65517/_x/c, core=collection1, node_name=127.0.0.1:65517__x%2Fc}
   [junit4]   2> 3165422 T8816 C1447 P65517 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:65517/_x/c/collection1/
   [junit4]   2> 3165423 T8809 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 3165423 T8816 C1447 P65517 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 3165423 T8816 C1447 P65517 oasc.SyncStrategy.syncToMe http://127.0.0.1:65517/_x/c/collection1/ has no replicas
   [junit4]   2> 3165423 T8816 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:65517/_x/c/collection1/ shard1
   [junit4]   2> 3165424 T8816 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 3165425 T8812 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3165425 T8812 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3165425 T8819 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3165428 T8809 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 3165430 T8812 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3165531 T8812 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3165531 T8819 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3165580 T8816 oasc.ZkController.register We are http://127.0.0.1:65517/_x/c/collection1/ and leader is http://127.0.0.1:65517/_x/c/collection1/
   [junit4]   2> 3165581 T8816 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:65517/_x/c
   [junit4]   2> 3165581 T8816 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 3165581 T8816 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
   [junit4]   2> 3165581 T8816 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 3165582 T8812 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3165583 T8816 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 3165583 T8821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3165583 T8820 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3165584 T8809 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 3165585 T8809 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:65517/_x/c",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:65517__x%2Fc",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 3165586 T8820 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3165688 T8820 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3165688 T8819 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3166592 T8786 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
   [junit4]   2> 3166593 T8786 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 3166595 T8786 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:63043
   [junit4]   2> 3166597 T8786 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 3166597 T8786 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3166597 T8786 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-003
   [junit4]   2> 3166598 T8786 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-003/'
   [junit4]   2> 3166635 T8786 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-003/solr.xml
   [junit4]   2> 3166690 T8786 oasc.CoreContainer.<init> New CoreContainer 1726624691
   [junit4]   2> 3166690 T8786 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-003/]
   [junit4]   2> 3166691 T8786 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 3166692 T8786 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 3166692 T8786 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 3166692 T8786 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 3166692 T8786 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 3166693 T8786 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 3166693 T8786 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 3166693 T8786 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 3166693 T8786 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 3166694 T8786 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 3166694 T8786 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3166695 T8786 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3166695 T8786 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 3166695 T8786 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:37683/solr
   [junit4]   2> 3166695 T8786 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 3166696 T8786 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3166698 T8832 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19e8618e name:ZooKeeperConnection Watcher:127.0.0.1:37683 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3166698 T8786 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3166701 T8786 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3166702 T8834 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7b42a3c7 name:ZooKeeperConnection Watcher:127.0.0.1:37683/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3166702 T8786 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3166707 T8786 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3167710 T8786 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63043__x%2Fc
   [junit4]   2> 3167711 T8786 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:63043__x%2Fc
   [junit4]   2> 3167722 T8836 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 3167722 T8836 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 3167723 T8820 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3167723 T8812 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3167723 T8836 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 3167723 T8821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3167724 T8809 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 3167725 T8809 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:63043/_x/c",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:63043__x%2Fc",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 3167725 T8809 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
   [junit4]   2> 3167725 T8809 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 3167727 T8821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3167727 T8835 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 3167727 T8821 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 3167727 T8819 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 3168724 T8836 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 3168725 T8836 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 3168726 T8836 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 3168726 T8836 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 3168727 T8836 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3168727 T8836 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-003/collection1/'
   [junit4]   2> 3168728 T8836 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-003/collection1/lib/classes/' to classloader
   [junit4]   2> 3168729 T8836 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-003/collection1/lib/README' to classloader
   [junit4]   2> 3168730 T8836 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-003/collection1/lib/.svn/' to classloader
   [junit4]   2> 3168850 T8836 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
   [junit4]   2> 3168901 T8836 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 3169003 T8836 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 3169016 T8836 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 3169461 T8836 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 3169478 T8836 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 3169482 T8836 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 3169493 T8836 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 3169498 T8836 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 3169503 T8836 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3169504 T8836 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 3169506 T8836 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 3169506 T8836 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3169507 T8836 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 3169507 T8836 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 3169507 T8836 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 3169508 T8836 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 3169508 T8836 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-003/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-003/collection1/data/
   [junit4]   2> 3169508 T8836 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5b4fab9a
   [junit4]   2> 3169509 T8836 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-003/collection1/data
   [junit4]   2> 3169510 T8836 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-003/collection1/data/index/
   [junit4]   2> 3169510 T8836 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-003/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 3169510 T8836 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-003/collection1/data/index
   [junit4]   2> 3169511 T8836 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=31, maxMergeAtOnceExplicit=15, maxMergedSegmentMB=46.5205078125, floorSegmentMB=0.3212890625, forceMergeDeletesPctAllowed=16.008820695250826, segmentsPerTier=46.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.2865870921699244
   [junit4]   2> 3169512 T8836 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@14c2ae6a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@63aadd07),segFN=segments_1,generation=1}
   [junit4]   2> 3169512 T8836 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 3169517 T8836 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 3169518 T8836 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 3169519 T8836 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 3169519 T8836 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3169519 T8836 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3169519 T8836 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 3169520 T8836 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3169521 T8836 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3169521 T8836 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 3169522 T8836 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 3169522 T8836 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 3169523 T8836 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 3169523 T8836 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 3169524 T8836 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 3169525 T8836 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 3169525 T8836 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 3169525 T8836 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 3169526 T8836 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 3169526 T8836 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 3169527 T8836 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 3169527 T8836 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 3169528 T8836 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 3169555 T8836 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 3169557 T8836 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 3169559 T8836 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 3169562 T8836 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 3169564 T8836 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 3169571 T8836 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 3169571 T8836 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 3169572 T8836 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=49, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=49.939453125, floorSegmentMB=1.9423828125, forceMergeDeletesPctAllowed=2.637825278493197, segmentsPerTier=45.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 3169573 T8836 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@14c2ae6a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@63aadd07),segFN=segments_1,generation=1}
   [junit4]   2> 3169573 T8836 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 3169573 T8836 oass.SolrIndexSearcher.<init> Opening Searcher@58c32608[collection1] main
   [junit4]   2> 3169574 T8836 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 3169575 T8836 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3169575 T8836 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 3169575 T8836 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 3169576 T8836 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 3169576 T8836 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3169577 T8836 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
   [junit4]   2> 3169577 T8836 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3169577 T8836 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
   [junit4]   2> 3169578 T8836 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 3169582 T8837 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@58c32608[collection1] main{UninvertingDirectoryReader()}
   [junit4]   2> 3169585 T8836 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 3169586 T8840 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:63043/_x/c collection:collection1 shard:shard1
   [junit4]   2> 3169586 T8786 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
   [junit4]   2> 3169587 T8786 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 3169587 T8840 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 3169591 T8840 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 3169592 T8821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3169592 T8840 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 3169592 T8840 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C1448 name=collection1 org.apache.solr.core.SolrCore@27a84924 url=http://127.0.0.1:63043/_x/c/collection1 node=127.0.0.1:63043__x%2Fc C1448_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:63043/_x/c, core=collection1, node_name=127.0.0.1:63043__x%2Fc}
   [junit4]   2> 3169592 T8840 C1448 P63043 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:63043/_x/c/collection1/
   [junit4]   2> 3169593 T8840 C1448 P63043 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 3169593 T8809 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 3169593 T8840 C1448 P63043 oasc.SyncStrategy.syncToMe http://127.0.0.1:63043/_x/c/collection1/ has no replicas
   [junit4]   2> 3169593 T8840 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:63043/_x/c/collection1/ shard1
   [junit4]   2> 3169593 T8840 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 3169595 T8821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3169595 T8821 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 3169595 T8819 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 3169595 T8835 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 3169597 T8809 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 3169599 T8821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3169701 T8821 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 3169701 T8835 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 3169701 T8819 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 3169750 T8840 oasc.ZkController.register We are http://127.0.0.1:63043/_x/c/collection1/ and leader is http://127.0.0.1:63043/_x/c/collection1/
   [junit4]   2> 3169751 T8840 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:63043/_x/c
   [junit4]   2> 3169751 T8840 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 3169751 T8840 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
   [junit4]   2> 3169751 T8840 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 3169752 T8821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3169752 T8812 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3169752 T8820 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3169753 T8840 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 3169753 T8809 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 3169754 T8809 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:63043/_x/c",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:63043__x%2Fc",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 3169755 T8820 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3169858 T8819 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 3169858 T8820 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 3169858 T8835 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 3171349 T8786 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
   [junit4]   2> 3171349 T8786 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 3171353 T8786 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:10100
   [junit4]   2> 3171353 T8786 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 3171353 T8786 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3171354 T8786 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-004
   [junit4]   2> 3171354 T8786 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-004/'
   [junit4]   2> 3171405 T8786 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-004/solr.xml
   [junit4]   2> 3171481 T8786 oasc.CoreContainer.<init> New CoreContainer 1256265506
   [junit4]   2> 3171482 T8786 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-004/]
   [junit4]   2> 3171483 T8786 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 3171483 T8786 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 3171483 T8786 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 3171484 T8786 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 3171484 T8786 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 3171484 T8786 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 3171484 T8786 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 3171485 T8786 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 3171485 T8786 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 3171485 T8786 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 3171486 T8786 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3171486 T8786 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3171486 T8786 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 3171486 T8786 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:37683/solr
   [junit4]   2> 3171487 T8786 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 3171488 T8786 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3171489 T8851 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7fb5df95 name:ZooKeeperConnection Watcher:127.0.0.1:37683 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3171490 T8786 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3171492 T8786 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3171493 T8853 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3d856391 name:ZooKeeperConnection Watcher:127.0.0.1:37683/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3171493 T8786 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3171497 T8786 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3172500 T8786 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:10100__x%2Fc
   [junit4]   2> 3172501 T8786 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:10100__x%2Fc
   [junit4]   2> 3172512 T8855 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 3172513 T8855 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 3172513 T8820 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3172513 T8812 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3172513 T8821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3172513 T8855 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 3172515 T8809 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 3172516 T8809 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:10100/_x/c",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:10100__x%2Fc",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 3172516 T8809 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
   [junit4]   2> 3172516 T8809 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 3172518 T8821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3172518 T8821 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 3172518 T8819 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 3172518 T8835 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 3172518 T8854 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 3173515 T8855 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 3173516 T8855 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 3173517 T8855 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 3173517 T8855 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 3173517 T8855 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3173518 T8855 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-004/collection1/'
   [junit4]   2> 3173519 T8855 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-004/collection1/lib/classes/' to classloader
   [junit4]   2> 3173520 T8855 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-004/collection1/lib/README' to classloader
   [junit4]   2> 3173520 T8855 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-004/collection1/lib/.svn/' to classloader
   [junit4]   2> 3173575 T8855 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
   [junit4]   2> 3173612 T8855 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 3173752 T8855 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 3173760 T8855 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 3174044 T8855 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 3174054 T8855 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 3174056 T8855 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 3174065 T8855 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 3174069 T8855 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 3174072 T8855 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3174073 T8855 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 3174074 T8855 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 3174075 T8855 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3174075 T8855 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 3174075 T8855 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 3174076 T8855 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 3174076 T8855 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 3174076 T8855 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-004/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-004/collection1/data/
   [junit4]   2> 3174076 T8855 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5b4fab9a
   [junit4]   2> 3174077 T8855 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-004/collection1/data
   [junit4]   2> 3174078 T8855 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-004/collection1/data/index/
   [junit4]   2> 3174078 T8855 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-004/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 3174078 T8855 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-004/collection1/data/index
   [junit4]   2> 3174078 T8855 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=31, maxMergeAtOnceExplicit=15, maxMergedSegmentMB=46.5205078125, floorSegmentMB=0.3212890625, forceMergeDeletesPctAllowed=16.008820695250826, segmentsPerTier=46.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.2865870921699244
   [junit4]   2> 3174079 T8855 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@57452f51 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6a47a5e8),segFN=segments_1,generation=1}
   [junit4]   2> 3174079 T8855 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 3174083 T8855 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 3174084 T8855 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 3174084 T8855 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 3174084 T8855 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3174084 T8855 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3174085 T8855 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 3174085 T8855 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3174085 T8855 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3174086 T8855 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 3174086 T8855 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 3174086 T8855 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 3174087 T8855 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 3174087 T8855 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 3174088 T8855 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 3174088 T8855 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 3174088 T8855 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 3174089 T8855 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 3174089 T8855 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 3174089 T8855 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 3174090 T8855 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 3174090 T8855 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 3174091 T8855 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 3174105 T8855 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 3174107 T8855 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 3174108 T8855 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 3174109 T8855 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 3174111 T8855 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 3174115 T8855 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 3174116 T8855 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 3174116 T8855 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=49, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=49.939453125, floorSegmentMB=1.9423828125, forceMergeDeletesPctAllowed=2.637825278493197, segmentsPerTier=45.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 3174117 T8855 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@57452f51 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6a47a5e8),segFN=segments_1,generation=1}
   [junit4]   2> 3174117 T8855 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 3174117 T8855 oass.SolrIndexSearcher.<init> Opening Searcher@6ebe415e[collection1] main
   [junit4]   2> 3174117 T8855 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 3174118 T8855 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3174118 T8855 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 3174119 T8855 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 3174119 T8855 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 3174119 T8855 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3174120 T8855 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
   [junit4]   2> 3174120 T8855 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3174120 T8855 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
   [junit4]   2> 3174120 T8855 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 3174123 T8856 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6ebe415e[collection1] main{UninvertingDirectoryReader()}
   [junit4]   2> 3174126 T8855 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 3174126 T8859 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:10100/_x/c collection:collection1 shard:shard2
   [junit4]   2> 3174127 T8786 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
   [junit4]   2> 3174127 T8786 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 3174127 T8859 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
   [junit4]   2> 3174131 T8859 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
   [junit4]   2> 3174132 T8821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3174132 T8859 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 3174132 T8859 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C1449 name=collection1 org.apache.solr.core.SolrCore@320c550b url=http://127.0.0.1:10100/_x/c/collection1 node=127.0.0.1:10100__x%2Fc C1449_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:10100/_x/c, core=collection1, node_name=127.0.0.1:10100__x%2Fc}
   [junit4]   2> 3174133 T8859 C1449 P10100 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:10100/_x/c/collection1/
   [junit4]   2> 3174133 T8809 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 3174133 T8859 C1449 P10100 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 3174134 T8859 C1449 P10100 oasc.SyncStrategy.syncToMe http://127.0.0.1:10100/_x/c/collection1/ has no replicas
   [junit4]   2> 3174134 T8859 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:10100/_x/c/collection1/ shard2
   [junit4]   2> 3174134 T8859 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
   [junit4]   2> 3174135 T8821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3174135 T8821 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 3174135 T8835 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 3174135 T8819 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 3174135 T8854 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 3174138 T8809 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 3174139 T8821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3174240 T8835 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 3174240 T8819 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 3174240 T8854 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 3174240 T8821 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 3174290 T8859 oasc.ZkController.register We are http://127.0.0.1:10100/_x/c/collection1/ and leader is http://127.0.0.1:10100/_x/c/collection1/
   [junit4]   2> 3174291 T8859 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:10100/_x/c
   [junit4]   2> 3174291 T8859 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 3174291 T8859 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
   [junit4]   2> 3174291 T8859 oasc.ZkController.publish numSh

[...truncated too long message...]

-001/tempDir-006/ocptest_shardsplit_shard3_replica1/data [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-006/ocptest_shardsplit_shard3_replica1/data;done=false>>]
   [junit4]   2> 3275946 T8786 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-006/ocptest_shardsplit_shard3_replica1/data
   [junit4]   2> 3275947 T10321 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:30546__x%2Fc
   [junit4]   2> 3275950 T10323 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:302)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:299)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:299)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:317)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:289)
   [junit4]   2> 
   [junit4]   2> 3277450 T10316 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 3277453 T8786 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/_x/c,null}
   [junit4]   2> 3280286 T8786 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
   [junit4]   2> 3280289 T8786 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:37683 37683
   [junit4]   2> 3280518 T8787 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:37683 37683
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=MultiThreadedOCPTest -Dtests.method=testDistribSearch -Dtests.seed=9BBB3C28E81C7420 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=bg -Dtests.timezone=Europe/Malta -Dtests.file.encoding=ISO-8859-1
   [junit4] FAILURE  119s J1 | MultiThreadedOCPTest.testDistribSearch <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: We have a failed SPLITSHARD task
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([9BBB3C28E81C7420:1A5DB2309F43141C]:0)
   [junit4]    > 	at org.apache.solr.cloud.MultiThreadedOCPTest.testTaskExclusivity(MultiThreadedOCPTest.java:125)
   [junit4]    > 	at org.apache.solr.cloud.MultiThreadedOCPTest.doTest(MultiThreadedOCPTest.java:71)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:865)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 3280546 T8786 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001
   [junit4]   2> 118560 T8785 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=DefaultSimilarity, locale=bg, timezone=Europe/Malta
   [junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_60 (64-bit)/cpus=16,threads=1,free=231981400,total=536870912
   [junit4]   2> NOTE: All tests run in this JVM: [SolrTestCaseJ4Test, TestSchemaNameResource, HdfsUnloadDistributedZkTest, TestSolr4Spatial2, TestCSVResponseWriter, TestStressLucene, TestFreeTextSuggestions, LoggingHandlerTest, TestFieldSortValues, DefaultValueUpdateProcessorTest, LeaderElectionIntegrationTest, TestConfigSets, CollectionsAPIDistributedZkTest, TestMergePolicyConfig, HdfsChaosMonkeySafeLeaderTest, AnalysisAfterCoreReloadTest, TestQueryUtils, TestExtendedDismaxParser, DirectUpdateHandlerOptimizeTest, TestCollapseQParserPlugin, UpdateRequestProcessorFactoryTest, CopyFieldTest, DocValuesMissingTest, TermVectorComponentTest, FullSolrCloudDistribCmdsTest, TestCloudSchemaless, TestSerializedLuceneMatchVersion, TestRestManager, TestSuggestSpellingConverter, TestDistributedMissingSort, TestPseudoReturnFields, ZkSolrClientTest, OpenCloseCoreStressTest, TestRandomMergePolicy, TestBinaryField, RankQueryTest, TestShardHandlerFactory, TestCollationField, TestIndexSearcher, AutoCommitTest, DistanceFunctionTest, CSVRequestHandlerTest, TestRealTimeGet, TestSolrXMLSerializer, TestLRUCache, DeleteReplicaTest, TestSolrXmlPersistor, TestReversedWildcardFilterFactory, SimplePostToolTest, DistributedQueryComponentOptimizationTest, DocValuesTest, TestCloudManagedSchema, TestHighFrequencyDictionaryFactory, UUIDUpdateProcessorFallbackTest, DistributedQueryElevationComponentTest, SynonymTokenizerTest, TestRecovery, TestNonNRTOpen, TestCursorMarkWithoutUniqueKey, IndexSchemaRuntimeFieldTest, DateMathParserTest, SOLR749Test, UniqFieldsUpdateProcessorFactoryTest, TestShortCircuitedRequests, MergeStrategyTest, SolrIndexConfigTest, SolrInfoMBeanTest, DistributedQueryComponentCustomSortTest, TestSolrQueryParserDefaultOperatorResource, TestMaxScoreQueryParser, TestGroupingSearch, DebugComponentTest, BinaryUpdateRequestHandlerTest, DistributedSuggestComponentTest, TestPostingsSolrHighlighter, CurrencyFieldXmlFileTest, TestIBSimilarityFactory, ScriptEngineTest, StressHdfsTest, TestFieldCollectionResource, TestPHPSerializedResponseWriter, FieldAnalysisRequestHandlerTest, ChaosMonkeySafeLeaderTest, HttpPartitionTest, SampleTest, ShardRoutingCustomTest, ConvertedLegacyTest, TestDistribDocBasedVersion, TestSystemIdResolver, TestCollectionAPI, HardAutoCommitTest, SpatialFilterTest, RangeFacetTest, HdfsRecoveryZkTest, SliceStateTest, SchemaVersionSpecificBehaviorTest, CacheHeaderTest, BasicZkTest, TestSweetSpotSimilarityFactory, SolrRequestParserTest, CursorPagingTest, BasicDistributedZk2Test, UnloadDistributedZkTest, ClusterStateUpdateTest, TestRandomFaceting, TestZkChroot, TestDistributedGrouping, TestFaceting, TestHashPartitioner, DistributedSpellCheckComponentTest, TermVectorComponentDistributedTest, TestJoin, TestMultiCoreConfBootstrap, TestReload, TestStressVersions, DistributedTermsComponentTest, SolrCoreTest, SpellCheckComponentTest, TestSort, TestFunctionQuery, TestLazyCores, DirectUpdateHandlerTest, TestBadConfig, OverseerCollectionProcessorTest, SoftAutoCommitTest, CurrencyFieldOpenExchangeTest, TestCoreDiscovery, SuggesterFSTTest, TestTrie, TestCSVLoader, NoCacheHeaderTest, TestWordDelimiterFilterFactory, DocValuesMultiTest, IndexBasedSpellCheckerTest, LukeRequestHandlerTest, TestQueryTypes, FileBasedSpellCheckerTest, TestValueSourceCache, TestSolrQueryParser, IndexSchemaTest, JSONWriterTest, QueryParsingTest, JsonLoaderTest, TestComponentsName, SearchHandlerTest, TestQuerySenderListener, UpdateParamsTest, TestQuerySenderNoQuery, TestSolrDeletionPolicy2, MultiTermTest, NumericFieldsTest, TestElisionMultitermQuery, TestConfig, OutputWriterTest, ExternalFileFieldSortTest, TestSolrCoreProperties, DirectSolrConnectionTest, NotRequiredUniqueKeyTest, TestLuceneMatchVersion, TestPhraseSuggestions, TestCharFilters, TestCodecSupport, TestXIncludeConfig, EchoParamsTest, TestDFRSimilarityFactory, TestPerFieldSimilarity, TestLMDirichletSimilarityFactory, TestBM25SimilarityFactory, TestDefaultSimilarityFactory, TestFastWriter, LegacyHTMLStripCharFilterTest, TestJmxMonitoredMap, ResourceLoaderTest, URLClassifyProcessorTest, TestFastLRUCache, ChaosMonkeyNothingIsSafeTest, CollectionsAPIAsyncDistributedZkTest, DeleteShardTest, DistribCursorPagingTest, MultiThreadedOCPTest]
   [junit4] Completed on J1 in 118.89s, 1 test, 1 failure <<< FAILURES!

[...truncated 226 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:474: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:454: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:45: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:496: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1298: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:922: There were test failures: 422 suites, 1761 tests, 1 failure, 53 ignored (19 assumptions)

Total time: 106 minutes 59 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-Tests-trunk-Java7 #4781
Archived 3 artifacts
Archive block size is 32768
Received 0 blocks and 228016853 bytes
Compression is 0.0%
Took 42 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure