You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/01/08 19:55:25 UTC

[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.7.0_45) - Build # 9016 - Failure!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/9016/
Java: 64bit/jdk1.7.0_45 -XX:+UseCompressedOops -XX:+UseParallelGC

2 tests failed.
REGRESSION:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch

Error Message:
There are still nodes recoverying - waited for 330 seconds

Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
	at __randomizedtesting.SeedInfo.seed([D6B497DE8AE3DF08:575219C6FDBCBF34]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:172)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:126)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:772)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:196)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:867)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:744)


REGRESSION:  org.apache.solr.cloud.OverseerTest.testOverseerFailure

Error Message:
KeeperErrorCode = NodeExists for /collections/collection1/leaders/shard1

Stack Trace:
org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /collections/collection1/leaders/shard1
	at __randomizedtesting.SeedInfo.seed([D6B497DE8AE3DF08:D2BC182D98463029]:0)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
	at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
	at org.apache.solr.common.cloud.SolrZkClient$10.execute(SolrZkClient.java:428)
	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:425)
	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:382)
	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:369)
	at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:112)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:107)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:155)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:275)
	at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:153)
	at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:584)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:744)




Build Log:
[...truncated 10334 lines...]
   [junit4] Suite: org.apache.solr.cloud.OverseerTest
   [junit4]   2> 614938 T1957 oas.SolrTestCaseJ4.initCore ####initCore
   [junit4]   2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-OverseerTest-1389206412122
   [junit4]   2> 614939 T1957 oas.SolrTestCaseJ4.initCore ####initCore end
   [junit4]   2> 614941 T1957 oas.SolrTestCaseJ4.setUp ###Starting testReplay
   [junit4]   2> 614942 T1957 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 614942 T1958 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 615042 T1957 oasc.ZkTestServer.run start zk server on port:38717
   [junit4]   2> 615043 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 615069 T1964 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5b0ff83a name:ZooKeeperConnection Watcher:127.0.0.1:38717/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 615069 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 615070 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 615072 T1966 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@73f419bd name:ZooKeeperConnection Watcher:127.0.0.1:38717 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 615072 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 615074 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 615075 T1968 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@64c7bcfb name:ZooKeeperConnection Watcher:127.0.0.1:38717 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 615075 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 615075 T1957 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 615077 T1957 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 615080 T1957 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 615081 T1957 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 615082 T1957 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 615084 T1957 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 615089 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 615091 T1970 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7e594b53 name:ZooKeeperConnection Watcher:127.0.0.1:38717/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 615091 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 615092 T1957 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 615093 T1957 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 615097 T1957 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 615098 T1957 oasc.Overseer.start Overseer (id=91043031425155075-127.0.0.1:38717_solr-n_0000000000) starting
   [junit4]   2> 615100 T1957 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 615103 T1957 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 615104 T1972 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 615106 T1971 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 615106 T1971 oasc.Overseer$ClusterStateUpdater.run Replaying operations from work queue.
   [junit4]   2> 615107 T1971 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "base_url":"http://127.0.0.1/solr",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "shard":"s1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "roles":"",
   [junit4]   2> 	  "state":"recovering"}
   [junit4]   2> 615107 T1964 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
   [junit4]   2> 615109 T1971 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "base_url":"http://127.0.0.1/solr",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "shard":"s1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "core":"core2",
   [junit4]   2> 	  "roles":"",
   [junit4]   2> 	  "state":"recovering"}
   [junit4]   2> 615110 T1964 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
   [junit4]   2> 615113 T1971 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 615116 T1971 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 615116 T1971 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "base_url":"http://127.0.0.1/solr",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "shard":"s1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "core":"core3",
   [junit4]   2> 	  "roles":"",
   [junit4]   2> 	  "state":"recovering"}
   [junit4]   2> 615122 T1970 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 615124 T1964 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
   [junit4]   2> 615208 T1957 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:38717 38717
   [junit4]   2> 615301 T1957 oas.SolrTestCaseJ4.tearDown ###Ending testReplay
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 615305 T1957 oas.SolrTestCaseJ4.setUp ###Starting testStateChange
   [junit4]   2> 615305 T1957 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 615305 T1973 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 615405 T1957 oasc.ZkTestServer.run start zk server on port:59834
   [junit4]   2> 615406 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 615409 T1979 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4bf2a90e name:ZooKeeperConnection Watcher:127.0.0.1:59834/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 615409 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 615409 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 615410 T1981 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3895fbd7 name:ZooKeeperConnection Watcher:127.0.0.1:59834 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 615411 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 615420 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 615421 T1983 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2fdb314e name:ZooKeeperConnection Watcher:127.0.0.1:59834 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 615422 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 615422 T1957 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 615423 T1957 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 615425 T1957 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 615427 T1957 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 615429 T1957 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 615431 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 615433 T1985 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4c79f1e9 name:ZooKeeperConnection Watcher:127.0.0.1:59834/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 615433 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 615434 T1957 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 615436 T1957 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 615439 T1957 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 615440 T1957 oasc.Overseer.start Overseer (id=91043031449010179-127.0.0.1:59834_solr-n_0000000000) starting
   [junit4]   2> 615442 T1957 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 615444 T1957 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 615446 T1957 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 615447 T1987 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 615448 T1986 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 615449 T1957 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 615450 T1986 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 615450 T1986 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "base_url":"http://127.0.0.1/solr",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "roles":"",
   [junit4]   2> 	  "state":"recovering"}
   [junit4]   2> 615450 T1986 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 615452 T1985 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 615452 T1979 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
   [junit4]   2> 615500 T1957 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 615503 T1985 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 615506 T1986 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 615507 T1986 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "base_url":"http://127.0.0.1/solr",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "roles":"",
   [junit4]   2> 	  "state":"active"}
   [junit4]   2> 615507 T1986 oasc.Overseer$ClusterStateUpdater.updateState node=core_node1 is already registered
   [junit4]   2> 615508 T1986 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
   [junit4]   2> 615512 T1985 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 615615 T1979 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
   [junit4]   2> 615656 T1957 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:59834 59834
   [junit4]   2> 615905 T1957 oas.SolrTestCaseJ4.tearDown ###Ending testStateChange
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 615908 T1957 oas.SolrTestCaseJ4.setUp ###Starting testShardAssignmentBigger
   [junit4]   2> 615909 T1957 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 615909 T1988 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 616009 T1957 oasc.ZkTestServer.run start zk server on port:60787
   [junit4]   2> 616010 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616012 T1994 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1258e653 name:ZooKeeperConnection Watcher:127.0.0.1:60787 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616013 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616026 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616029 T1996 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5595e2a4 name:ZooKeeperConnection Watcher:127.0.0.1:60787 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616030 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616030 T1957 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 616034 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616035 T1998 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3e7c7bd4 name:ZooKeeperConnection Watcher:127.0.0.1:60787/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616036 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616036 T1957 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 616040 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616041 T2000 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6ecc321 name:ZooKeeperConnection Watcher:127.0.0.1:60787/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616042 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616043 T1957 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 616045 T1957 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 616051 T1957 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 616054 T1957 oasc.Overseer.start Overseer (id=91043031488593923-127.0.0.1:60787_solr-n_0000000000) starting
   [junit4]   2> 616057 T1957 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 616063 T1957 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 616067 T1957 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 616070 T2002 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 616070 T1957 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 616071 T2001 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 616073 T1957 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 616075 T1957 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 616078 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616080 T2004 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@10398a6a name:ZooKeeperConnection Watcher:127.0.0.1:60787/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616080 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616081 T1957 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 616082 T1957 oascc.SolrZkClient.makePath makePath: /live_nodes/node0
   [junit4]   2> 616084 T1998 oascc.ZkStateReader$3.process Updating live nodes... (1)
   [junit4]   2> 616084 T2004 oascc.ZkStateReader$3.process Updating live nodes... (1)
   [junit4]   2> 616085 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616086 T2006 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@253098c0 name:ZooKeeperConnection Watcher:127.0.0.1:60787/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616086 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616087 T1957 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 616089 T1957 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
   [junit4]   2> 616090 T2004 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 616090 T1998 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 616090 T2006 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 616091 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616092 T2008 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4d40df1e name:ZooKeeperConnection Watcher:127.0.0.1:60787/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616092 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616092 T1957 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 616094 T1957 oascc.SolrZkClient.makePath makePath: /live_nodes/node2
   [junit4]   2> 616095 T1998 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 616095 T2006 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 616096 T2008 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 616095 T2004 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 616096 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616097 T2010 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4d775ce6 name:ZooKeeperConnection Watcher:127.0.0.1:60787/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616097 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616098 T1957 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 616099 T1957 oascc.SolrZkClient.makePath makePath: /live_nodes/node3
   [junit4]   2> 616101 T2006 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 616101 T2004 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 616101 T1998 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 616101 T2008 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 616101 T2010 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 616101 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616102 T2012 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4b8e61bc name:ZooKeeperConnection Watcher:127.0.0.1:60787/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616102 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616103 T1957 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 616104 T1957 oascc.SolrZkClient.makePath makePath: /live_nodes/node4
   [junit4]   2> 616106 T2010 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 616106 T1998 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 616106 T2004 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 616106 T2006 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 616106 T2012 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 616106 T2008 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 616107 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616108 T2014 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3e6c19be name:ZooKeeperConnection Watcher:127.0.0.1:60787/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616108 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616108 T1957 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 616110 T1957 oascc.SolrZkClient.makePath makePath: /live_nodes/node5
   [junit4]   2> 616111 T2008 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 616112 T2004 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 616112 T2014 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 616112 T2012 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 616112 T1998 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 616112 T2010 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 616112 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616112 T2006 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 616113 T2016 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@76d258cf name:ZooKeeperConnection Watcher:127.0.0.1:60787/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616113 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616114 T1957 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 616115 T1957 oascc.SolrZkClient.makePath makePath: /live_nodes/node6
   [junit4]   2> 616117 T2010 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 616117 T2004 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 616117 T2008 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 616118 T2006 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 616118 T2012 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 616118 T1998 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 616118 T2014 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 616118 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616118 T2016 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 616119 T2018 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@60a3ca4d name:ZooKeeperConnection Watcher:127.0.0.1:60787/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616119 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616120 T1957 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 616121 T1957 oascc.SolrZkClient.makePath makePath: /live_nodes/node7
   [junit4]   2> 616123 T1998 oascc.ZkStateReader$3.process Updating live nodes... (8)
   [junit4]   2> 616123 T2008 oascc.ZkStateReader$3.process Updating live nodes... (8)
   [junit4]   2> 616123 T2014 oascc.ZkStateReader$3.process Updating live nodes... (8)
   [junit4]   2> 616124 T2010 oascc.ZkStateReader$3.process Updating live nodes... (8)
   [junit4]   2> 616124 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616124 T2006 oascc.ZkStateReader$3.process Updating live nodes... (8)
   [junit4]   2> 616124 T2016 oascc.ZkStateReader$3.process Updating live nodes... (8)
   [junit4]   2> 616124 T2012 oascc.ZkStateReader$3.process Updating live nodes... (8)
   [junit4]   2> 616123 T2018 oascc.ZkStateReader$3.process Updating live nodes... (8)
   [junit4]   2> 616124 T2004 oascc.ZkStateReader$3.process Updating live nodes... (8)
   [junit4]   2> 616125 T2020 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e12f4a6 name:ZooKeeperConnection Watcher:127.0.0.1:60787/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616125 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616126 T1957 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 616127 T1957 oascc.SolrZkClient.makePath makePath: /live_nodes/node8
   [junit4]   2> 616129 T2016 oascc.ZkStateReader$3.process Updating live nodes... (9)
   [junit4]   2> 616129 T2006 oascc.ZkStateReader$3.process Updating live nodes... (9)
   [junit4]   2> 616130 T2014 oascc.ZkStateReader$3.process Updating live nodes... (9)
   [junit4]   2> 616130 T2012 oascc.ZkStateReader$3.process Updating live nodes... (9)
   [junit4]   2> 616130 T2010 oascc.ZkStateReader$3.process Updating live nodes... (9)
   [junit4]   2> 616129 T2004 oascc.ZkStateReader$3.process Updating live nodes... (9)
   [junit4]   2> 616130 T2018 oascc.ZkStateReader$3.process Updating live nodes... (9)
   [junit4]   2> 616130 T2020 oascc.ZkStateReader$3.process Updating live nodes... (9)
   [junit4]   2> 616130 T2008 oascc.ZkStateReader$3.process Updating live nodes... (9)
   [junit4]   2> 616130 T1998 oascc.ZkStateReader$3.process Updating live nodes... (9)
   [junit4]   2> 616130 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616131 T2022 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@728a8c5c name:ZooKeeperConnection Watcher:127.0.0.1:60787/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616132 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616133 T1957 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 616135 T1957 oascc.SolrZkClient.makePath makePath: /live_nodes/node9
   [junit4]   2> 616137 T2014 oascc.ZkStateReader$3.process Updating live nodes... (10)
   [junit4]   2> 616138 T2010 oascc.ZkStateReader$3.process Updating live nodes... (10)
   [junit4]   2> 616137 T2004 oascc.ZkStateReader$3.process Updating live nodes... (10)
   [junit4]   2> 616137 T2006 oascc.ZkStateReader$3.process Updating live nodes... (10)
   [junit4]   2> 616138 T2018 oascc.ZkStateReader$3.process Updating live nodes... (10)
   [junit4]   2> 616138 T2022 oascc.ZkStateReader$3.process Updating live nodes... (10)
   [junit4]   2> 616138 T2016 oascc.ZkStateReader$3.process Updating live nodes... (10)
   [junit4]   2> 616138 T1998 oascc.ZkStateReader$3.process Updating live nodes... (10)
   [junit4]   2> 616138 T2020 oascc.ZkStateReader$3.process Updating live nodes... (10)
   [junit4]   2> 616138 T2012 oascc.ZkStateReader$3.process Updating live nodes... (10)
   [junit4]   2> 616138 T2008 oascc.ZkStateReader$3.process Updating live nodes... (10)
   [junit4]   2> 616138 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616140 T2024 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@627ff5f5 name:ZooKeeperConnection Watcher:127.0.0.1:60787/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616140 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616141 T1957 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 616144 T1957 oascc.SolrZkClient.makePath makePath: /live_nodes/node10
   [junit4]   2> 616145 T2010 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 616146 T2004 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 616146 T2018 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 616146 T2014 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 616146 T1998 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 616146 T2020 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 616147 T2024 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 616146 T2008 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 616147 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616146 T2022 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 616146 T2012 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 616146 T2006 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 616146 T2016 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 616148 T2026 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2b2ef473 name:ZooKeeperConnection Watcher:127.0.0.1:60787/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616148 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616149 T1957 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 616150 T1957 oascc.SolrZkClient.makePath makePath: /live_nodes/node11
   [junit4]   2> 616153 T2024 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 616153 T2018 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 616153 T2006 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 616153 T2020 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 616153 T2012 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 616153 T2014 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 616154 T2022 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 616154 T1998 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 616154 T2004 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 616153 T2008 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 616154 T2026 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 616154 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616154 T2016 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 616154 T2010 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 616155 T2028 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7c043e11 name:ZooKeeperConnection Watcher:127.0.0.1:60787/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616155 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616156 T1957 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 616157 T1957 oascc.SolrZkClient.makePath makePath: /live_nodes/node12
   [junit4]   2> 616160 T2020 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 616160 T2006 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 616161 T2014 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 616161 T2016 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 616161 T2012 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 616161 T1998 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 616160 T2010 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 616160 T2008 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 616160 T2018 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 616161 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616161 T2026 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 616161 T2028 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 616161 T2024 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 616161 T2004 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 616161 T2022 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 616163 T2030 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@144cf091 name:ZooKeeperConnection Watcher:127.0.0.1:60787/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616163 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616164 T1957 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 616165 T1957 oascc.SolrZkClient.makePath makePath: /live_nodes/node13
   [junit4]   2> 616167 T2028 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 616167 T2018 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 616167 T2024 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 616168 T2020 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 616168 T2012 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 616168 T1998 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 616168 T2010 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 616168 T2014 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 616168 T2026 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 616168 T2030 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 616168 T2006 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 616168 T2016 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 616168 T2022 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 616168 T2004 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 616168 T2008 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 616169 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616170 T2032 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2893b5c name:ZooKeeperConnection Watcher:127.0.0.1:60787/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616170 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616171 T1957 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 616172 T1957 oascc.SolrZkClient.makePath makePath: /live_nodes/node14
   [junit4]   2> 616174 T2020 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 616175 T2006 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 616175 T2012 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 616175 T2032 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 616175 T2010 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 616175 T2016 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 616175 T2014 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 616175 T2030 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 616175 T2018 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 616175 T1998 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 616175 T2004 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 616175 T2024 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 616176 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616176 T2028 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 616175 T2008 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 616175 T2022 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 616175 T2026 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 616177 T2034 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@219f8c8c name:ZooKeeperConnection Watcher:127.0.0.1:60787/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616177 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616178 T1957 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 616181 T1957 oascc.SolrZkClient.makePath makePath: /live_nodes/node15
   [junit4]   2> 616184 T2004 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 616184 T2032 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 616184 T2010 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 616184 T2016 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 616184 T2022 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 616185 T2030 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 616185 T2018 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 616185 T2014 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 616185 T2024 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 616185 T1998 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 616185 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616184 T2008 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 616184 T2020 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 616184 T2026 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 616185 T2034 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 616185 T2028 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 616185 T2012 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 616185 T2006 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 616186 T2036 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3def47bf name:ZooKeeperConnection Watcher:127.0.0.1:60787/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616187 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616187 T1957 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 616189 T1957 oascc.SolrZkClient.makePath makePath: /live_nodes/node16
   [junit4]   2> 616191 T2012 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 616192 T2024 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 616191 T2014 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 616192 T2006 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 616191 T2020 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 616193 T2026 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 616191 T2028 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 616193 T2034 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 616193 T2036 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 616193 T1998 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 616193 T2022 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 616192 T2018 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 616192 T2016 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 616192 T2008 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 616192 T2030 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 616192 T2004 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 616192 T2032 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 616192 T2010 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 616193 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616194 T2038 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3471fcbf name:ZooKeeperConnection Watcher:127.0.0.1:60787/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616195 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616195 T1957 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 616197 T1957 oascc.SolrZkClient.makePath makePath: /live_nodes/node17
   [junit4]   2> 616199 T2028 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 616199 T2024 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 616199 T2018 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 616200 T2006 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 616200 T2012 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 616200 T2014 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 616200 T2020 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 616200 T2008 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 616200 T2036 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 616201 T2004 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 616200 T2034 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 616200 T2022 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 616200 T2030 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 616200 T1998 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 616201 T2026 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 616201 T2038 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 616201 T2010 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 616201 T2032 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 616201 T2016 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 616201 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616202 T2040 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5238ac02 name:ZooKeeperConnection Watcher:127.0.0.1:60787/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616202 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616203 T1957 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 616204 T1957 oascc.SolrZkClient.makePath makePath: /live_nodes/node18
   [junit4]   2> 616207 T2004 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 616207 T2010 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 616207 T2016 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 616207 T2022 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 616207 T1998 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 616207 T2026 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 616208 T2024 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 616208 T2028 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 616207 T2034 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 616208 T2020 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 616209 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616208 T2018 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 616208 T2012 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 616208 T2036 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 616208 T2006 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 616208 T2014 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 616208 T2030 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 616208 T2040 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 616208 T2032 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 616208 T2008 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 616210 T2042 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@72253947 name:ZooKeeperConnection Watcher:127.0.0.1:60787/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616209 T2038 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 616210 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616211 T1957 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 616212 T1957 oascc.SolrZkClient.makePath makePath: /live_nodes/node19
   [junit4]   2> 616215 T2024 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 616215 T2012 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 616215 T2020 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 616215 T2028 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 616215 T2018 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 616216 T2030 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 616216 T2022 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 616216 T2034 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 616216 T2014 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 616216 T2006 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 616216 T2036 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 616216 T2008 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 616217 T2042 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 616217 T2032 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 616217 T2026 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 616217 T2004 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 616217 T1998 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 616216 T2016 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 616217 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616217 T2040 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 616220 T2044 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6ac8c805 name:ZooKeeperConnection Watcher:127.0.0.1:60787/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616220 T2038 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 616220 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616220 T2010 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 616221 T1957 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 616223 T1957 oascc.SolrZkClient.makePath makePath: /live_nodes/node20
   [junit4]   2> 616226 T2042 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 616226 T2014 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 616227 T2020 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 616227 T2006 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 616227 T2012 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 616227 T2030 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 616226 T2024 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 616227 T2008 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 616227 T2028 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 616227 T2022 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 616228 T2034 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 616228 T2016 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 616227 T2018 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 616228 T2004 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 616228 T2036 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 616228 T2040 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 616228 T1998 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 616228 T2038 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 616227 T2044 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 616228 T2026 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 616228 T2032 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 616228 T2010 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 616229 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616230 T2046 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ff07791 name:ZooKeeperConnection Watcher:127.0.0.1:60787/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616230 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616231 T1957 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 616232 T1957 oascc.SolrZkClient.makePath makePath: /live_nodes/node21
   [junit4]   2> 616235 T2016 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 616235 T2026 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 616235 T2004 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 616235 T2022 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 616235 T2036 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 616236 T2006 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 616236 T2030 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 616236 T2012 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 616236 T2038 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 616236 T2046 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 616235 T2014 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 616235 T2032 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 616235 T2010 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 616237 T2034 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 616236 T2042 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 616236 T2024 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 616236 T2028 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 616236 T2018 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 616236 T2044 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 616236 T2040 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 616236 T2020 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 616236 T2008 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 616236 T1998 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 616237 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616238 T2048 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@454a162 name:ZooKeeperConnection Watcher:127.0.0.1:60787/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616238 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616239 T1957 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 616240 T1957 oascc.SolrZkClient.makePath makePath: /live_nodes/node22
   [junit4]   2> 616243 T2004 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 616243 T2010 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 616244 T2026 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 616244 T2032 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 616244 T2040 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 616245 T2018 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 616244 T2016 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 616243 T2036 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 616245 T2028 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 616245 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616245 T2042 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 616245 T2048 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 616244 T2020 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 616244 T2038 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 616244 T2024 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 616244 T2012 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 616244 T2044 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 616244 T2008 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 616244 T2030 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 616244 T2006 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 616244 T2014 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 616244 T2034 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 616244 T2022 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 616246 T2050 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5643a267 name:ZooKeeperConnection Watcher:127.0.0.1:60787/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616247 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616245 T2046 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 616245 T1998 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 616248 T1957 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 616250 T1957 oascc.SolrZkClient.makePath makePath: /live_nodes/node23
   [junit4]   2> 616253 T2010 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 616253 T2026 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 616253 T2040 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 616254 T2008 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 616253 T2016 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 616254 T2020 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 616254 T2014 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 616254 T2032 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 616254 T2036 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 616254 T2044 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 616254 T2030 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 616254 T2012 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 616254 T2046 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 616255 T2024 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 616253 T1998 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 616253 T2048 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 616255 T2050 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 616255 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616255 T2028 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 616255 T2042 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 616255 T2034 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 616255 T2022 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 616254 T2018 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 616254 T2004 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 616254 T2038 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 616254 T2006 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 616257 T2052 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6979f129 name:ZooKeeperConnection Watcher:127.0.0.1:60787/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616257 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616258 T1957 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 616259 T1957 oascc.SolrZkClient.makePath makePath: /live_nodes/node24
   [junit4]   2> 616261 T2048 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 616261 T2038 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 616262 T2040 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 616262 T2008 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 616262 T2022 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 616262 T2046 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 616262 T1998 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 616263 T2028 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 616262 T2032 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 616263 T2042 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 616263 T2014 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 616263 T2050 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 616263 T2030 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 616263 T2006 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 616263 T2010 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 616263 T2044 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 616263 T2016 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 616263 T2012 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 616263 T2024 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 616262 T2036 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 616262 T2026 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 616262 T2020 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 616265 T2054 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@32a7eecf name:ZooKeeperConnection Watcher:127.0.0.1:60787/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616262 T2004 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 616262 T2018 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 616264 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616263 T2052 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 616263 T2034 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 616265 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616266 T1957 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 616268 T1957 oascc.SolrZkClient.makePath makePath: /live_nodes/node25
   [junit4]   2> 616271 T2034 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 616271 T2038 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 616271 T2032 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 616272 T2052 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 616272 T2036 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 616272 T2040 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 616272 T1998 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 616271 T2022 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 616272 T2008 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 616272 T2048 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 616272 T2026 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 616273 T2010 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 616271 T2004 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 616273 T2020 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 616273 T2014 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 616273 T2016 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 616273 T2018 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 616272 T2054 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 616272 T2044 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 616272 T2046 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 616272 T2024 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 616272 T2042 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 616272 T2028 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 616272 T2050 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 616273 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616273 T2030 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 616273 T2012 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 616273 T2006 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 616275 T2056 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@23ec0e52 name:ZooKeeperConnection Watcher:127.0.0.1:60787/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616275 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616276 T1957 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 616277 T1957 oascc.SolrZkClient.makePath makePath: /live_nodes/node26
   [junit4]   2> 616280 T2034 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 616280 T2036 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 616281 T2052 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 616280 T2038 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 616281 T2028 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 616281 T2048 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 616281 T2054 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 616281 T2026 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 616281 T2012 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 616282 T2006 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 616281 T2008 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 616281 T2022 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 616281 T2046 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 616281 T1998 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 616281 T2020 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 616281 T2040 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 616281 T2010 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 616281 T2024 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 616281 T2004 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 616280 T2042 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 616282 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616282 T2016 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 616282 T2050 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 616283 T2058 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15d178c0 name:ZooKeeperConnection Watcher:127.0.0.1:60787/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616282 T2032 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 616282 T2014 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 616282 T2056 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 616281 T2044 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 616281 T2030 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 616281 T2018 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 616284 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616285 T1957 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 616286 T1957 oascc.SolrZkClient.makePath makePath: /live_nodes/node27
   [junit4]   2> 616289 T2018 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 616290 T2030 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 616290 T2020 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 616290 T2010 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 616290 T2006 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 616289 T2008 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 616290 T2056 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 616290 T1998 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 616291 T2038 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 616290 T2012 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 616290 T2046 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 616290 T2054 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 616291 T2004 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 616290 T2014 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 616291 T2026 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 616292 T2022 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 616290 T2048 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 616290 T2044 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 616292 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616292 T2058 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 616292 T2028 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 616291 T2032 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 616291 T2016 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 616291 T2050 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 616291 T2036 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 616291 T2042 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 616291 T2024 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 616291 T2052 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 616291 T2034 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 616291 T2040 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 616293 T2060 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b585d5c name:ZooKeeperConnection Watcher:127.0.0.1:60787/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616294 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616294 T1957 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 616296 T1957 oascc.SolrZkClient.makePath makePath: /live_nodes/node28
   [junit4]   2> 616298 T2052 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 616300 T2018 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 616300 T2020 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 616300 T2010 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 616302 T2060 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 616300 T2034 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 616300 T2016 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 616300 T2024 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 616299 T2006 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 616299 T2022 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 616299 T1998 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 616299 T2014 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 616299 T2008 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 616299 T2026 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 616299 T2028 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 616299 T2036 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 616299 T2042 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 616299 T2004 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 616299 T2050 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 616299 T2038 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 616299 T2032 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 616303 T2062 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1738ead8 name:ZooKeeperConnection Watcher:127.0.0.1:60787/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 616302 T2012 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 616302 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616302 T2046 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 616302 T2030 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 616301 T2048 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 616301 T2044 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 616301 T2056 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 616301 T2054 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 616301 T2058 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 616301 T2040 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 616304 T1957 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 616306 T1957 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 616308 T1957 oascc.SolrZkClient.makePath makePath: /live_nodes/node29
   [junit4]   2> 616311 T2040 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 616311 T2062 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 616312 T2026 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 616312 T2042 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 616313 T2038 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 616313 T2052 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 616313 T2004 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 616312 T2032 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 616312 T2020 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 616312 T2018 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 616313 T1957 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 616312 T2034 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 616312 T2022 oascc.ZkStateReader$3.proce

[...truncated too long message...]

.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:170)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:170)
   [junit4]   2> 	at org.apache.solr.cloud.ElectionContext.cancelElection(ElectionContext.java:67)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:276)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:107)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:136)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 897348 T925 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 897348 T925 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:253)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:253)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:91)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:136)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 897452 T939 C1850 P45655 oasc.SyncStrategy.sync WARN Closed, skipping sync up.
   [junit4]   2> 897452 T939 oasc.ShardLeaderElectionContext.rejoinLeaderElection Not rejoining election because CoreContainer is shutdown
   [junit4]   2> 897452 T939 oasc.SolrCore.close [awholynewcollection_0_shard1_replica2]  CLOSING SolrCore org.apache.solr.core.SolrCore@3d735809
   [junit4]   2> 897461 T939 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=307,transaction_logs_total_number=1}
   [junit4]   2> 897461 T939 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 897461 T939 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 897461 T939 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 897462 T939 oasc.SolrCore.closeSearcher [awholynewcollection_0_shard1_replica2] Closing main searcher on request.
   [junit4]   2> 897462 T939 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 897462 T939 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1389206278783/awholynewcollection_0_shard1_replica2/data/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1389206278783/awholynewcollection_0_shard1_replica2/data/index;done=false>>]
   [junit4]   2> 897463 T939 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1389206278783/awholynewcollection_0_shard1_replica2/data/index
   [junit4]   2> 897463 T939 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1389206278783/awholynewcollection_0_shard1_replica2/data [CachedDir<<refCount=0;path=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1389206278783/awholynewcollection_0_shard1_replica2/data;done=false>>]
   [junit4]   2> 897463 T939 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1389206278783/awholynewcollection_0_shard1_replica2/data
   [junit4]   2> 897463 T939 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 897463 T939 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:253)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:253)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:91)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:136)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> NOTE: test params are: codec=Lucene46: {range_facet_si=PostingsFormat(name=Memory doPackFST= true), range_facet_l=PostingsFormat(name=Memory doPackFST= true), a_t=PostingsFormat(name=Memory doPackFST= true), a_si=FST41, range_facet_sl=MockFixedIntBlock(blockSize=1376), intDefault=FST41, multiDefault=PostingsFormat(name=Memory doPackFST= true), _version_=FST41, timestamp=PostingsFormat(name=Memory doPackFST= true), other_tl1=FST41, id=MockFixedIntBlock(blockSize=1376), text=PostingsFormat(name=Direct)}, docValues:{timestamp=DocValuesFormat(name=Lucene45)}, sim=DefaultSimilarity, locale=ca_ES, timezone=America/North_Dakota/Beulah
   [junit4]   2> NOTE: Linux 3.8.0-35-generic amd64/Oracle Corporation 1.7.0_45 (64-bit)/cpus=8,threads=1,free=150270488,total=398458880
   [junit4]   2> NOTE: All tests run in this JVM: [AssignTest, DocValuesMultiTest, DocValuesMissingTest, BasicZkTest, TestArbitraryIndexDir, DocumentAnalysisRequestHandlerTest, TestStressRecovery, TestStressUserVersions, IndexBasedSpellCheckerTest, DisMaxRequestHandlerTest, DistribCursorPagingTest, BlockDirectoryTest, CoreContainerCoreInitFailuresTest, TestDefaultSimilarityFactory, TestSolrJ, SolrIndexConfigTest, TestDistributedGrouping, AlternateDirectoryTest, SpellCheckComponentTest, NoCacheHeaderTest, CurrencyFieldXmlFileTest, MultiTermTest, PluginInfoTest, TestRandomMergePolicy, TestSolrQueryParserDefaultOperatorResource, PreAnalyzedUpdateProcessorTest, TestCoreDiscovery, TestFuzzyAnalyzedSuggestions, TestManagedSchemaFieldResource, PingRequestHandlerTest, TestFieldResource, RequestHandlersTest, TestCSVLoader, TestAddFieldRealTimeGet, TestQuerySenderListener, TestFastWriter, TestSweetSpotSimilarityFactory, TestStressLucene, SpellingQueryConverterTest, TimeZoneUtilsTest, TestBinaryResponseWriter, CursorMarkTest, TestSolrCoreProperties, TestRecovery, TestDefaultSearchFieldResource, CurrencyFieldOpenExchangeTest, DateFieldTest, TestRTGBase, TestPhraseSuggestions, DefaultValueUpdateProcessorTest, CollectionsAPIDistributedZkTest]
   [junit4] Completed on J1 in 420.26s, 1 test, 1 failure <<< FAILURES!

[...truncated 672 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:453: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:433: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:491: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1304: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:937: There were test failures: 362 suites, 1586 tests, 1 error, 1 failure, 34 ignored (6 assumptions)

Total time: 44 minutes 43 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0_45 -XX:+UseCompressedOops -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.7.0_60-ea-b02) - Build # 9018 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/9018/
Java: 64bit/jdk1.7.0_60-ea-b02 -XX:+UseCompressedOops -XX:+UseSerialGC

6 tests failed.
REGRESSION:  org.apache.solr.analytics.expression.ExpressionTest.negateTest

Error Message:
Caught exception in getStatResult, xPath = /response/lst[@name='stats']/lst[@name='nr']/double[@name='sum']  raw data: <?xml version="1.0" encoding="UTF-8"?> <response>  <lst name="responseHeader">   <int name="status">0</int>   <int name="QTime">12</int> </lst> <result name="response" numFound="100" start="0"> </result> <lst name="stats">   <lst name="stats_fields"/> </lst> <lst name="stats">   <lst name="rr">     <str name="max">str9</str>     <str name="min">str0</str>     <str name="rmax">9rts</str>     <str name="rmin">0rts</str>   </lst>   <lst name="dmr">     <str name="cma">+2MONTHS</str>     <str name="cme">+2YEARS</str>     <date name="dmma">1812-02-29T23:59:59Z</date>     <date name="dmme">1807-12-31T23:59:59Z</date>     <date name="max">1811-12-31T23:59:59Z</date>     <date name="median">1805-12-31T23:59:59Z</date>   </lst>   <lst name="cr">     <str name="ccmax">this is the secondstr9</str>     <str name="ccmin">this is the firststr0</str>     <str name="csmax">this is the second</str>     <str name="csmin">this is the first</str>     <str name="max">str9</str>     <str name="min">str0</str>   </lst>   <lst name="mr">     <long name="count">100</long>     <null name="mcm"/>     <null name="mean"/>     <null name="median"/>     <null name="su"/>     <null name="sum"/>     <null name="unique"/>   </lst>   <lst name="nr">     <double name="c">-100.0</double>     <long name="count">100</long>     <null name="s"/>     <null name="sum"/>   </lst>   <lst name="avr">     <double name="c">100.0</double>     <long name="count">100</long>     <null name="s"/>     <null name="sum"/>   </lst>   <lst name="cnr">     <double name="c10">10.0</double>     <double name="c8">8.0</double>   </lst>   <lst name="pr">     <long name="count">100</long>     <null name="mc"/>     <null name="mean"/>     <null name="su"/>     <null name="sum"/>     <null name="unique"/>   </lst>   <lst name="cdr">     <date name="cd1">1800-12-31T23:59:59Z</date>     <date name="cd2">1804-06-30T23:59:59Z</date>     <str name="cs1">1800-12-31T23:59:59Z</str>     <str name="cs2">1804-06-30T23:59:59Z</str>   </lst>   <lst name="csr">     <str name="cs1">this is the first</str>     <str name="cs2">this is the second</str>     <str name="cs3">this is the third</str>   </lst>   <lst name="dr">     <long name="count">100</long>     <null name="mc"/>     <null name="mean"/>     <null name="su"/>     <null name="sum"/>     <null name="unique"/>   </lst>   <lst name="ar">     <long name="count">100</long>     <null name="mcm"/>     <null name="mean"/>     <null name="median"/>     <null name="su"/>     <null name="sum"/>     <null name="unique"/>   </lst> </lst> </response> 

Stack Trace:
java.lang.AssertionError: Caught exception in getStatResult, xPath = /response/lst[@name='stats']/lst[@name='nr']/double[@name='sum'] 
raw data: <?xml version="1.0" encoding="UTF-8"?>
<response>

<lst name="responseHeader">
  <int name="status">0</int>
  <int name="QTime">12</int>
</lst>
<result name="response" numFound="100" start="0">
</result>
<lst name="stats">
  <lst name="stats_fields"/>
</lst>
<lst name="stats">
  <lst name="rr">
    <str name="max">str9</str>
    <str name="min">str0</str>
    <str name="rmax">9rts</str>
    <str name="rmin">0rts</str>
  </lst>
  <lst name="dmr">
    <str name="cma">+2MONTHS</str>
    <str name="cme">+2YEARS</str>
    <date name="dmma">1812-02-29T23:59:59Z</date>
    <date name="dmme">1807-12-31T23:59:59Z</date>
    <date name="max">1811-12-31T23:59:59Z</date>
    <date name="median">1805-12-31T23:59:59Z</date>
  </lst>
  <lst name="cr">
    <str name="ccmax">this is the secondstr9</str>
    <str name="ccmin">this is the firststr0</str>
    <str name="csmax">this is the second</str>
    <str name="csmin">this is the first</str>
    <str name="max">str9</str>
    <str name="min">str0</str>
  </lst>
  <lst name="mr">
    <long name="count">100</long>
    <null name="mcm"/>
    <null name="mean"/>
    <null name="median"/>
    <null name="su"/>
    <null name="sum"/>
    <null name="unique"/>
  </lst>
  <lst name="nr">
    <double name="c">-100.0</double>
    <long name="count">100</long>
    <null name="s"/>
    <null name="sum"/>
  </lst>
  <lst name="avr">
    <double name="c">100.0</double>
    <long name="count">100</long>
    <null name="s"/>
    <null name="sum"/>
  </lst>
  <lst name="cnr">
    <double name="c10">10.0</double>
    <double name="c8">8.0</double>
  </lst>
  <lst name="pr">
    <long name="count">100</long>
    <null name="mc"/>
    <null name="mean"/>
    <null name="su"/>
    <null name="sum"/>
    <null name="unique"/>
  </lst>
  <lst name="cdr">
    <date name="cd1">1800-12-31T23:59:59Z</date>
    <date name="cd2">1804-06-30T23:59:59Z</date>
    <str name="cs1">1800-12-31T23:59:59Z</str>
    <str name="cs2">1804-06-30T23:59:59Z</str>
  </lst>
  <lst name="csr">
    <str name="cs1">this is the first</str>
    <str name="cs2">this is the second</str>
    <str name="cs3">this is the third</str>
  </lst>
  <lst name="dr">
    <long name="count">100</long>
    <null name="mc"/>
    <null name="mean"/>
    <null name="su"/>
    <null name="sum"/>
    <null name="unique"/>
  </lst>
  <lst name="ar">
    <long name="count">100</long>
    <null name="mcm"/>
    <null name="mean"/>
    <null name="median"/>
    <null name="su"/>
    <null name="sum"/>
    <null name="unique"/>
  </lst>
</lst>
</response>

	at __randomizedtesting.SeedInfo.seed([3880E4F26B38C358:30888C57CF1593E8]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.analytics.AbstractAnalyticsStatsTest.getStatResult(AbstractAnalyticsStatsTest.java:113)
	at org.apache.solr.analytics.expression.ExpressionTest.negateTest(ExpressionTest.java:132)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:744)


REGRESSION:  org.apache.solr.analytics.expression.ExpressionTest.multiplyTest

Error Message:
Caught exception in getStatResult, xPath = /response/lst[@name='stats']/lst[@name='mr']/double[@name='sum']  raw data: <?xml version="1.0" encoding="UTF-8"?> <response>  <lst name="responseHeader">   <int name="status">0</int>   <int name="QTime">12</int> </lst> <result name="response" numFound="100" start="0"> </result> <lst name="stats">   <lst name="stats_fields"/> </lst> <lst name="stats">   <lst name="rr">     <str name="max">str9</str>     <str name="min">str0</str>     <str name="rmax">9rts</str>     <str name="rmin">0rts</str>   </lst>   <lst name="dmr">     <str name="cma">+2MONTHS</str>     <str name="cme">+2YEARS</str>     <date name="dmma">1812-02-29T23:59:59Z</date>     <date name="dmme">1807-12-31T23:59:59Z</date>     <date name="max">1811-12-31T23:59:59Z</date>     <date name="median">1805-12-31T23:59:59Z</date>   </lst>   <lst name="cr">     <str name="ccmax">this is the secondstr9</str>     <str name="ccmin">this is the firststr0</str>     <str name="csmax">this is the second</str>     <str name="csmin">this is the first</str>     <str name="max">str9</str>     <str name="min">str0</str>   </lst>   <lst name="mr">     <long name="count">100</long>     <null name="mcm"/>     <null name="mean"/>     <null name="median"/>     <null name="su"/>     <null name="sum"/>     <null name="unique"/>   </lst>   <lst name="nr">     <double name="c">-100.0</double>     <long name="count">100</long>     <null name="s"/>     <null name="sum"/>   </lst>   <lst name="avr">     <double name="c">100.0</double>     <long name="count">100</long>     <null name="s"/>     <null name="sum"/>   </lst>   <lst name="cnr">     <double name="c10">10.0</double>     <double name="c8">8.0</double>   </lst>   <lst name="pr">     <long name="count">100</long>     <null name="mc"/>     <null name="mean"/>     <null name="su"/>     <null name="sum"/>     <null name="unique"/>   </lst>   <lst name="cdr">     <date name="cd1">1800-12-31T23:59:59Z</date>     <date name="cd2">1804-06-30T23:59:59Z</date>     <str name="cs1">1800-12-31T23:59:59Z</str>     <str name="cs2">1804-06-30T23:59:59Z</str>   </lst>   <lst name="csr">     <str name="cs1">this is the first</str>     <str name="cs2">this is the second</str>     <str name="cs3">this is the third</str>   </lst>   <lst name="dr">     <long name="count">100</long>     <null name="mc"/>     <null name="mean"/>     <null name="su"/>     <null name="sum"/>     <null name="unique"/>   </lst>   <lst name="ar">     <long name="count">100</long>     <null name="mcm"/>     <null name="mean"/>     <null name="median"/>     <null name="su"/>     <null name="sum"/>     <null name="unique"/>   </lst> </lst> </response> 

Stack Trace:
java.lang.AssertionError: Caught exception in getStatResult, xPath = /response/lst[@name='stats']/lst[@name='mr']/double[@name='sum'] 
raw data: <?xml version="1.0" encoding="UTF-8"?>
<response>

<lst name="responseHeader">
  <int name="status">0</int>
  <int name="QTime">12</int>
</lst>
<result name="response" numFound="100" start="0">
</result>
<lst name="stats">
  <lst name="stats_fields"/>
</lst>
<lst name="stats">
  <lst name="rr">
    <str name="max">str9</str>
    <str name="min">str0</str>
    <str name="rmax">9rts</str>
    <str name="rmin">0rts</str>
  </lst>
  <lst name="dmr">
    <str name="cma">+2MONTHS</str>
    <str name="cme">+2YEARS</str>
    <date name="dmma">1812-02-29T23:59:59Z</date>
    <date name="dmme">1807-12-31T23:59:59Z</date>
    <date name="max">1811-12-31T23:59:59Z</date>
    <date name="median">1805-12-31T23:59:59Z</date>
  </lst>
  <lst name="cr">
    <str name="ccmax">this is the secondstr9</str>
    <str name="ccmin">this is the firststr0</str>
    <str name="csmax">this is the second</str>
    <str name="csmin">this is the first</str>
    <str name="max">str9</str>
    <str name="min">str0</str>
  </lst>
  <lst name="mr">
    <long name="count">100</long>
    <null name="mcm"/>
    <null name="mean"/>
    <null name="median"/>
    <null name="su"/>
    <null name="sum"/>
    <null name="unique"/>
  </lst>
  <lst name="nr">
    <double name="c">-100.0</double>
    <long name="count">100</long>
    <null name="s"/>
    <null name="sum"/>
  </lst>
  <lst name="avr">
    <double name="c">100.0</double>
    <long name="count">100</long>
    <null name="s"/>
    <null name="sum"/>
  </lst>
  <lst name="cnr">
    <double name="c10">10.0</double>
    <double name="c8">8.0</double>
  </lst>
  <lst name="pr">
    <long name="count">100</long>
    <null name="mc"/>
    <null name="mean"/>
    <null name="su"/>
    <null name="sum"/>
    <null name="unique"/>
  </lst>
  <lst name="cdr">
    <date name="cd1">1800-12-31T23:59:59Z</date>
    <date name="cd2">1804-06-30T23:59:59Z</date>
    <str name="cs1">1800-12-31T23:59:59Z</str>
    <str name="cs2">1804-06-30T23:59:59Z</str>
  </lst>
  <lst name="csr">
    <str name="cs1">this is the first</str>
    <str name="cs2">this is the second</str>
    <str name="cs3">this is the third</str>
  </lst>
  <lst name="dr">
    <long name="count">100</long>
    <null name="mc"/>
    <null name="mean"/>
    <null name="su"/>
    <null name="sum"/>
    <null name="unique"/>
  </lst>
  <lst name="ar">
    <long name="count">100</long>
    <null name="mcm"/>
    <null name="mean"/>
    <null name="median"/>
    <null name="su"/>
    <null name="sum"/>
    <null name="unique"/>
  </lst>
</lst>
</response>

	at __randomizedtesting.SeedInfo.seed([3880E4F26B38C358:5301FA8C5E36F536]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.analytics.AbstractAnalyticsStatsTest.getStatResult(AbstractAnalyticsStatsTest.java:113)
	at org.apache.solr.analytics.expression.ExpressionTest.multiplyTest(ExpressionTest.java:92)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:744)


REGRESSION:  org.apache.solr.analytics.expression.ExpressionTest.absoluteValueTest

Error Message:
Caught exception in getStatResult, xPath = /response/lst[@name='stats']/lst[@name='avr']/double[@name='sum']  raw data: <?xml version="1.0" encoding="UTF-8"?> <response>  <lst name="responseHeader">   <int name="status">0</int>   <int name="QTime">12</int> </lst> <result name="response" numFound="100" start="0"> </result> <lst name="stats">   <lst name="stats_fields"/> </lst> <lst name="stats">   <lst name="rr">     <str name="max">str9</str>     <str name="min">str0</str>     <str name="rmax">9rts</str>     <str name="rmin">0rts</str>   </lst>   <lst name="dmr">     <str name="cma">+2MONTHS</str>     <str name="cme">+2YEARS</str>     <date name="dmma">1812-02-29T23:59:59Z</date>     <date name="dmme">1807-12-31T23:59:59Z</date>     <date name="max">1811-12-31T23:59:59Z</date>     <date name="median">1805-12-31T23:59:59Z</date>   </lst>   <lst name="cr">     <str name="ccmax">this is the secondstr9</str>     <str name="ccmin">this is the firststr0</str>     <str name="csmax">this is the second</str>     <str name="csmin">this is the first</str>     <str name="max">str9</str>     <str name="min">str0</str>   </lst>   <lst name="mr">     <long name="count">100</long>     <null name="mcm"/>     <null name="mean"/>     <null name="median"/>     <null name="su"/>     <null name="sum"/>     <null name="unique"/>   </lst>   <lst name="nr">     <double name="c">-100.0</double>     <long name="count">100</long>     <null name="s"/>     <null name="sum"/>   </lst>   <lst name="avr">     <double name="c">100.0</double>     <long name="count">100</long>     <null name="s"/>     <null name="sum"/>   </lst>   <lst name="cnr">     <double name="c10">10.0</double>     <double name="c8">8.0</double>   </lst>   <lst name="pr">     <long name="count">100</long>     <null name="mc"/>     <null name="mean"/>     <null name="su"/>     <null name="sum"/>     <null name="unique"/>   </lst>   <lst name="cdr">     <date name="cd1">1800-12-31T23:59:59Z</date>     <date name="cd2">1804-06-30T23:59:59Z</date>     <str name="cs1">1800-12-31T23:59:59Z</str>     <str name="cs2">1804-06-30T23:59:59Z</str>   </lst>   <lst name="csr">     <str name="cs1">this is the first</str>     <str name="cs2">this is the second</str>     <str name="cs3">this is the third</str>   </lst>   <lst name="dr">     <long name="count">100</long>     <null name="mc"/>     <null name="mean"/>     <null name="su"/>     <null name="sum"/>     <null name="unique"/>   </lst>   <lst name="ar">     <long name="count">100</long>     <null name="mcm"/>     <null name="mean"/>     <null name="median"/>     <null name="su"/>     <null name="sum"/>     <null name="unique"/>   </lst> </lst> </response> 

Stack Trace:
java.lang.AssertionError: Caught exception in getStatResult, xPath = /response/lst[@name='stats']/lst[@name='avr']/double[@name='sum'] 
raw data: <?xml version="1.0" encoding="UTF-8"?>
<response>

<lst name="responseHeader">
  <int name="status">0</int>
  <int name="QTime">12</int>
</lst>
<result name="response" numFound="100" start="0">
</result>
<lst name="stats">
  <lst name="stats_fields"/>
</lst>
<lst name="stats">
  <lst name="rr">
    <str name="max">str9</str>
    <str name="min">str0</str>
    <str name="rmax">9rts</str>
    <str name="rmin">0rts</str>
  </lst>
  <lst name="dmr">
    <str name="cma">+2MONTHS</str>
    <str name="cme">+2YEARS</str>
    <date name="dmma">1812-02-29T23:59:59Z</date>
    <date name="dmme">1807-12-31T23:59:59Z</date>
    <date name="max">1811-12-31T23:59:59Z</date>
    <date name="median">1805-12-31T23:59:59Z</date>
  </lst>
  <lst name="cr">
    <str name="ccmax">this is the secondstr9</str>
    <str name="ccmin">this is the firststr0</str>
    <str name="csmax">this is the second</str>
    <str name="csmin">this is the first</str>
    <str name="max">str9</str>
    <str name="min">str0</str>
  </lst>
  <lst name="mr">
    <long name="count">100</long>
    <null name="mcm"/>
    <null name="mean"/>
    <null name="median"/>
    <null name="su"/>
    <null name="sum"/>
    <null name="unique"/>
  </lst>
  <lst name="nr">
    <double name="c">-100.0</double>
    <long name="count">100</long>
    <null name="s"/>
    <null name="sum"/>
  </lst>
  <lst name="avr">
    <double name="c">100.0</double>
    <long name="count">100</long>
    <null name="s"/>
    <null name="sum"/>
  </lst>
  <lst name="cnr">
    <double name="c10">10.0</double>
    <double name="c8">8.0</double>
  </lst>
  <lst name="pr">
    <long name="count">100</long>
    <null name="mc"/>
    <null name="mean"/>
    <null name="su"/>
    <null name="sum"/>
    <null name="unique"/>
  </lst>
  <lst name="cdr">
    <date name="cd1">1800-12-31T23:59:59Z</date>
    <date name="cd2">1804-06-30T23:59:59Z</date>
    <str name="cs1">1800-12-31T23:59:59Z</str>
    <str name="cs2">1804-06-30T23:59:59Z</str>
  </lst>
  <lst name="csr">
    <str name="cs1">this is the first</str>
    <str name="cs2">this is the second</str>
    <str name="cs3">this is the third</str>
  </lst>
  <lst name="dr">
    <long name="count">100</long>
    <null name="mc"/>
    <null name="mean"/>
    <null name="su"/>
    <null name="sum"/>
    <null name="unique"/>
  </lst>
  <lst name="ar">
    <long name="count">100</long>
    <null name="mcm"/>
    <null name="mean"/>
    <null name="median"/>
    <null name="su"/>
    <null name="sum"/>
    <null name="unique"/>
  </lst>
</lst>
</response>

	at __randomizedtesting.SeedInfo.seed([3880E4F26B38C358:192CC4E7A6A59B3E]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.analytics.AbstractAnalyticsStatsTest.getStatResult(AbstractAnalyticsStatsTest.java:113)
	at org.apache.solr.analytics.expression.ExpressionTest.absoluteValueTest(ExpressionTest.java:143)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:744)


REGRESSION:  org.apache.solr.analytics.expression.ExpressionTest.powerTest

Error Message:
Caught exception in getStatResult, xPath = /response/lst[@name='stats']/lst[@name='pr']/double[@name='sum']  raw data: <?xml version="1.0" encoding="UTF-8"?> <response>  <lst name="responseHeader">   <int name="status">0</int>   <int name="QTime">12</int> </lst> <result name="response" numFound="100" start="0"> </result> <lst name="stats">   <lst name="stats_fields"/> </lst> <lst name="stats">   <lst name="rr">     <str name="max">str9</str>     <str name="min">str0</str>     <str name="rmax">9rts</str>     <str name="rmin">0rts</str>   </lst>   <lst name="dmr">     <str name="cma">+2MONTHS</str>     <str name="cme">+2YEARS</str>     <date name="dmma">1812-02-29T23:59:59Z</date>     <date name="dmme">1807-12-31T23:59:59Z</date>     <date name="max">1811-12-31T23:59:59Z</date>     <date name="median">1805-12-31T23:59:59Z</date>   </lst>   <lst name="cr">     <str name="ccmax">this is the secondstr9</str>     <str name="ccmin">this is the firststr0</str>     <str name="csmax">this is the second</str>     <str name="csmin">this is the first</str>     <str name="max">str9</str>     <str name="min">str0</str>   </lst>   <lst name="mr">     <long name="count">100</long>     <null name="mcm"/>     <null name="mean"/>     <null name="median"/>     <null name="su"/>     <null name="sum"/>     <null name="unique"/>   </lst>   <lst name="nr">     <double name="c">-100.0</double>     <long name="count">100</long>     <null name="s"/>     <null name="sum"/>   </lst>   <lst name="avr">     <double name="c">100.0</double>     <long name="count">100</long>     <null name="s"/>     <null name="sum"/>   </lst>   <lst name="cnr">     <double name="c10">10.0</double>     <double name="c8">8.0</double>   </lst>   <lst name="pr">     <long name="count">100</long>     <null name="mc"/>     <null name="mean"/>     <null name="su"/>     <null name="sum"/>     <null name="unique"/>   </lst>   <lst name="cdr">     <date name="cd1">1800-12-31T23:59:59Z</date>     <date name="cd2">1804-06-30T23:59:59Z</date>     <str name="cs1">1800-12-31T23:59:59Z</str>     <str name="cs2">1804-06-30T23:59:59Z</str>   </lst>   <lst name="csr">     <str name="cs1">this is the first</str>     <str name="cs2">this is the second</str>     <str name="cs3">this is the third</str>   </lst>   <lst name="dr">     <long name="count">100</long>     <null name="mc"/>     <null name="mean"/>     <null name="su"/>     <null name="sum"/>     <null name="unique"/>   </lst>   <lst name="ar">     <long name="count">100</long>     <null name="mcm"/>     <null name="mean"/>     <null name="median"/>     <null name="su"/>     <null name="sum"/>     <null name="unique"/>   </lst> </lst> </response> 

Stack Trace:
java.lang.AssertionError: Caught exception in getStatResult, xPath = /response/lst[@name='stats']/lst[@name='pr']/double[@name='sum'] 
raw data: <?xml version="1.0" encoding="UTF-8"?>
<response>

<lst name="responseHeader">
  <int name="status">0</int>
  <int name="QTime">12</int>
</lst>
<result name="response" numFound="100" start="0">
</result>
<lst name="stats">
  <lst name="stats_fields"/>
</lst>
<lst name="stats">
  <lst name="rr">
    <str name="max">str9</str>
    <str name="min">str0</str>
    <str name="rmax">9rts</str>
    <str name="rmin">0rts</str>
  </lst>
  <lst name="dmr">
    <str name="cma">+2MONTHS</str>
    <str name="cme">+2YEARS</str>
    <date name="dmma">1812-02-29T23:59:59Z</date>
    <date name="dmme">1807-12-31T23:59:59Z</date>
    <date name="max">1811-12-31T23:59:59Z</date>
    <date name="median">1805-12-31T23:59:59Z</date>
  </lst>
  <lst name="cr">
    <str name="ccmax">this is the secondstr9</str>
    <str name="ccmin">this is the firststr0</str>
    <str name="csmax">this is the second</str>
    <str name="csmin">this is the first</str>
    <str name="max">str9</str>
    <str name="min">str0</str>
  </lst>
  <lst name="mr">
    <long name="count">100</long>
    <null name="mcm"/>
    <null name="mean"/>
    <null name="median"/>
    <null name="su"/>
    <null name="sum"/>
    <null name="unique"/>
  </lst>
  <lst name="nr">
    <double name="c">-100.0</double>
    <long name="count">100</long>
    <null name="s"/>
    <null name="sum"/>
  </lst>
  <lst name="avr">
    <double name="c">100.0</double>
    <long name="count">100</long>
    <null name="s"/>
    <null name="sum"/>
  </lst>
  <lst name="cnr">
    <double name="c10">10.0</double>
    <double name="c8">8.0</double>
  </lst>
  <lst name="pr">
    <long name="count">100</long>
    <null name="mc"/>
    <null name="mean"/>
    <null name="su"/>
    <null name="sum"/>
    <null name="unique"/>
  </lst>
  <lst name="cdr">
    <date name="cd1">1800-12-31T23:59:59Z</date>
    <date name="cd2">1804-06-30T23:59:59Z</date>
    <str name="cs1">1800-12-31T23:59:59Z</str>
    <str name="cs2">1804-06-30T23:59:59Z</str>
  </lst>
  <lst name="csr">
    <str name="cs1">this is the first</str>
    <str name="cs2">this is the second</str>
    <str name="cs3">this is the third</str>
  </lst>
  <lst name="dr">
    <long name="count">100</long>
    <null name="mc"/>
    <null name="mean"/>
    <null name="su"/>
    <null name="sum"/>
    <null name="unique"/>
  </lst>
  <lst name="ar">
    <long name="count">100</long>
    <null name="mcm"/>
    <null name="mean"/>
    <null name="median"/>
    <null name="su"/>
    <null name="sum"/>
    <null name="unique"/>
  </lst>
</lst>
</response>

	at __randomizedtesting.SeedInfo.seed([3880E4F26B38C358:563253F1832F9D89]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.analytics.AbstractAnalyticsStatsTest.getStatResult(AbstractAnalyticsStatsTest.java:113)
	at org.apache.solr.analytics.expression.ExpressionTest.powerTest(ExpressionTest.java:119)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:744)


REGRESSION:  org.apache.solr.analytics.expression.ExpressionTest.addTest

Error Message:
Caught exception in getStatResult, xPath = /response/lst[@name='stats']/lst[@name='ar']/double[@name='sum']  raw data: <?xml version="1.0" encoding="UTF-8"?> <response>  <lst name="responseHeader">   <int name="status">0</int>   <int name="QTime">12</int> </lst> <result name="response" numFound="100" start="0"> </result> <lst name="stats">   <lst name="stats_fields"/> </lst> <lst name="stats">   <lst name="rr">     <str name="max">str9</str>     <str name="min">str0</str>     <str name="rmax">9rts</str>     <str name="rmin">0rts</str>   </lst>   <lst name="dmr">     <str name="cma">+2MONTHS</str>     <str name="cme">+2YEARS</str>     <date name="dmma">1812-02-29T23:59:59Z</date>     <date name="dmme">1807-12-31T23:59:59Z</date>     <date name="max">1811-12-31T23:59:59Z</date>     <date name="median">1805-12-31T23:59:59Z</date>   </lst>   <lst name="cr">     <str name="ccmax">this is the secondstr9</str>     <str name="ccmin">this is the firststr0</str>     <str name="csmax">this is the second</str>     <str name="csmin">this is the first</str>     <str name="max">str9</str>     <str name="min">str0</str>   </lst>   <lst name="mr">     <long name="count">100</long>     <null name="mcm"/>     <null name="mean"/>     <null name="median"/>     <null name="su"/>     <null name="sum"/>     <null name="unique"/>   </lst>   <lst name="nr">     <double name="c">-100.0</double>     <long name="count">100</long>     <null name="s"/>     <null name="sum"/>   </lst>   <lst name="avr">     <double name="c">100.0</double>     <long name="count">100</long>     <null name="s"/>     <null name="sum"/>   </lst>   <lst name="cnr">     <double name="c10">10.0</double>     <double name="c8">8.0</double>   </lst>   <lst name="pr">     <long name="count">100</long>     <null name="mc"/>     <null name="mean"/>     <null name="su"/>     <null name="sum"/>     <null name="unique"/>   </lst>   <lst name="cdr">     <date name="cd1">1800-12-31T23:59:59Z</date>     <date name="cd2">1804-06-30T23:59:59Z</date>     <str name="cs1">1800-12-31T23:59:59Z</str>     <str name="cs2">1804-06-30T23:59:59Z</str>   </lst>   <lst name="csr">     <str name="cs1">this is the first</str>     <str name="cs2">this is the second</str>     <str name="cs3">this is the third</str>   </lst>   <lst name="dr">     <long name="count">100</long>     <null name="mc"/>     <null name="mean"/>     <null name="su"/>     <null name="sum"/>     <null name="unique"/>   </lst>   <lst name="ar">     <long name="count">100</long>     <null name="mcm"/>     <null name="mean"/>     <null name="median"/>     <null name="su"/>     <null name="sum"/>     <null name="unique"/>   </lst> </lst> </response> 

Stack Trace:
java.lang.AssertionError: Caught exception in getStatResult, xPath = /response/lst[@name='stats']/lst[@name='ar']/double[@name='sum'] 
raw data: <?xml version="1.0" encoding="UTF-8"?>
<response>

<lst name="responseHeader">
  <int name="status">0</int>
  <int name="QTime">12</int>
</lst>
<result name="response" numFound="100" start="0">
</result>
<lst name="stats">
  <lst name="stats_fields"/>
</lst>
<lst name="stats">
  <lst name="rr">
    <str name="max">str9</str>
    <str name="min">str0</str>
    <str name="rmax">9rts</str>
    <str name="rmin">0rts</str>
  </lst>
  <lst name="dmr">
    <str name="cma">+2MONTHS</str>
    <str name="cme">+2YEARS</str>
    <date name="dmma">1812-02-29T23:59:59Z</date>
    <date name="dmme">1807-12-31T23:59:59Z</date>
    <date name="max">1811-12-31T23:59:59Z</date>
    <date name="median">1805-12-31T23:59:59Z</date>
  </lst>
  <lst name="cr">
    <str name="ccmax">this is the secondstr9</str>
    <str name="ccmin">this is the firststr0</str>
    <str name="csmax">this is the second</str>
    <str name="csmin">this is the first</str>
    <str name="max">str9</str>
    <str name="min">str0</str>
  </lst>
  <lst name="mr">
    <long name="count">100</long>
    <null name="mcm"/>
    <null name="mean"/>
    <null name="median"/>
    <null name="su"/>
    <null name="sum"/>
    <null name="unique"/>
  </lst>
  <lst name="nr">
    <double name="c">-100.0</double>
    <long name="count">100</long>
    <null name="s"/>
    <null name="sum"/>
  </lst>
  <lst name="avr">
    <double name="c">100.0</double>
    <long name="count">100</long>
    <null name="s"/>
    <null name="sum"/>
  </lst>
  <lst name="cnr">
    <double name="c10">10.0</double>
    <double name="c8">8.0</double>
  </lst>
  <lst name="pr">
    <long name="count">100</long>
    <null name="mc"/>
    <null name="mean"/>
    <null name="su"/>
    <null name="sum"/>
    <null name="unique"/>
  </lst>
  <lst name="cdr">
    <date name="cd1">1800-12-31T23:59:59Z</date>
    <date name="cd2">1804-06-30T23:59:59Z</date>
    <str name="cs1">1800-12-31T23:59:59Z</str>
    <str name="cs2">1804-06-30T23:59:59Z</str>
  </lst>
  <lst name="csr">
    <str name="cs1">this is the first</str>
    <str name="cs2">this is the second</str>
    <str name="cs3">this is the third</str>
  </lst>
  <lst name="dr">
    <long name="count">100</long>
    <null name="mc"/>
    <null name="mean"/>
    <null name="su"/>
    <null name="sum"/>
    <null name="unique"/>
  </lst>
  <lst name="ar">
    <long name="count">100</long>
    <null name="mcm"/>
    <null name="mean"/>
    <null name="median"/>
    <null name="su"/>
    <null name="sum"/>
    <null name="unique"/>
  </lst>
</lst>
</response>

	at __randomizedtesting.SeedInfo.seed([3880E4F26B38C358:CEAC492C2AF7C41F]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.analytics.AbstractAnalyticsStatsTest.getStatResult(AbstractAnalyticsStatsTest.java:113)
	at org.apache.solr.analytics.expression.ExpressionTest.addTest(ExpressionTest.java:78)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:744)


REGRESSION:  org.apache.solr.analytics.expression.ExpressionTest.divideTest

Error Message:
Caught exception in getStatResult, xPath = /response/lst[@name='stats']/lst[@name='dr']/double[@name='sum']  raw data: <?xml version="1.0" encoding="UTF-8"?> <response>  <lst name="responseHeader">   <int name="status">0</int>   <int name="QTime">12</int> </lst> <result name="response" numFound="100" start="0"> </result> <lst name="stats">   <lst name="stats_fields"/> </lst> <lst name="stats">   <lst name="rr">     <str name="max">str9</str>     <str name="min">str0</str>     <str name="rmax">9rts</str>     <str name="rmin">0rts</str>   </lst>   <lst name="dmr">     <str name="cma">+2MONTHS</str>     <str name="cme">+2YEARS</str>     <date name="dmma">1812-02-29T23:59:59Z</date>     <date name="dmme">1807-12-31T23:59:59Z</date>     <date name="max">1811-12-31T23:59:59Z</date>     <date name="median">1805-12-31T23:59:59Z</date>   </lst>   <lst name="cr">     <str name="ccmax">this is the secondstr9</str>     <str name="ccmin">this is the firststr0</str>     <str name="csmax">this is the second</str>     <str name="csmin">this is the first</str>     <str name="max">str9</str>     <str name="min">str0</str>   </lst>   <lst name="mr">     <long name="count">100</long>     <null name="mcm"/>     <null name="mean"/>     <null name="median"/>     <null name="su"/>     <null name="sum"/>     <null name="unique"/>   </lst>   <lst name="nr">     <double name="c">-100.0</double>     <long name="count">100</long>     <null name="s"/>     <null name="sum"/>   </lst>   <lst name="avr">     <double name="c">100.0</double>     <long name="count">100</long>     <null name="s"/>     <null name="sum"/>   </lst>   <lst name="cnr">     <double name="c10">10.0</double>     <double name="c8">8.0</double>   </lst>   <lst name="pr">     <long name="count">100</long>     <null name="mc"/>     <null name="mean"/>     <null name="su"/>     <null name="sum"/>     <null name="unique"/>   </lst>   <lst name="cdr">     <date name="cd1">1800-12-31T23:59:59Z</date>     <date name="cd2">1804-06-30T23:59:59Z</date>     <str name="cs1">1800-12-31T23:59:59Z</str>     <str name="cs2">1804-06-30T23:59:59Z</str>   </lst>   <lst name="csr">     <str name="cs1">this is the first</str>     <str name="cs2">this is the second</str>     <str name="cs3">this is the third</str>   </lst>   <lst name="dr">     <long name="count">100</long>     <null name="mc"/>     <null name="mean"/>     <null name="su"/>     <null name="sum"/>     <null name="unique"/>   </lst>   <lst name="ar">     <long name="count">100</long>     <null name="mcm"/>     <null name="mean"/>     <null name="median"/>     <null name="su"/>     <null name="sum"/>     <null name="unique"/>   </lst> </lst> </response> 

Stack Trace:
java.lang.AssertionError: Caught exception in getStatResult, xPath = /response/lst[@name='stats']/lst[@name='dr']/double[@name='sum'] 
raw data: <?xml version="1.0" encoding="UTF-8"?>
<response>

<lst name="responseHeader">
  <int name="status">0</int>
  <int name="QTime">12</int>
</lst>
<result name="response" numFound="100" start="0">
</result>
<lst name="stats">
  <lst name="stats_fields"/>
</lst>
<lst name="stats">
  <lst name="rr">
    <str name="max">str9</str>
    <str name="min">str0</str>
    <str name="rmax">9rts</str>
    <str name="rmin">0rts</str>
  </lst>
  <lst name="dmr">
    <str name="cma">+2MONTHS</str>
    <str name="cme">+2YEARS</str>
    <date name="dmma">1812-02-29T23:59:59Z</date>
    <date name="dmme">1807-12-31T23:59:59Z</date>
    <date name="max">1811-12-31T23:59:59Z</date>
    <date name="median">1805-12-31T23:59:59Z</date>
  </lst>
  <lst name="cr">
    <str name="ccmax">this is the secondstr9</str>
    <str name="ccmin">this is the firststr0</str>
    <str name="csmax">this is the second</str>
    <str name="csmin">this is the first</str>
    <str name="max">str9</str>
    <str name="min">str0</str>
  </lst>
  <lst name="mr">
    <long name="count">100</long>
    <null name="mcm"/>
    <null name="mean"/>
    <null name="median"/>
    <null name="su"/>
    <null name="sum"/>
    <null name="unique"/>
  </lst>
  <lst name="nr">
    <double name="c">-100.0</double>
    <long name="count">100</long>
    <null name="s"/>
    <null name="sum"/>
  </lst>
  <lst name="avr">
    <double name="c">100.0</double>
    <long name="count">100</long>
    <null name="s"/>
    <null name="sum"/>
  </lst>
  <lst name="cnr">
    <double name="c10">10.0</double>
    <double name="c8">8.0</double>
  </lst>
  <lst name="pr">
    <long name="count">100</long>
    <null name="mc"/>
    <null name="mean"/>
    <null name="su"/>
    <null name="sum"/>
    <null name="unique"/>
  </lst>
  <lst name="cdr">
    <date name="cd1">1800-12-31T23:59:59Z</date>
    <date name="cd2">1804-06-30T23:59:59Z</date>
    <str name="cs1">1800-12-31T23:59:59Z</str>
    <str name="cs2">1804-06-30T23:59:59Z</str>
  </lst>
  <lst name="csr">
    <str name="cs1">this is the first</str>
    <str name="cs2">this is the second</str>
    <str name="cs3">this is the third</str>
  </lst>
  <lst name="dr">
    <long name="count">100</long>
    <null name="mc"/>
    <null name="mean"/>
    <null name="su"/>
    <null name="sum"/>
    <null name="unique"/>
  </lst>
  <lst name="ar">
    <long name="count">100</long>
    <null name="mcm"/>
    <null name="mean"/>
    <null name="median"/>
    <null name="su"/>
    <null name="sum"/>
    <null name="unique"/>
  </lst>
</lst>
</response>

	at __randomizedtesting.SeedInfo.seed([3880E4F26B38C358:608D0F044FFAA18E]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.analytics.AbstractAnalyticsStatsTest.getStatResult(AbstractAnalyticsStatsTest.java:113)
	at org.apache.solr.analytics.expression.ExpressionTest.divideTest(ExpressionTest.java:106)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:744)




Build Log:
[...truncated 10349 lines...]
   [junit4] Suite: org.apache.solr.analytics.expression.ExpressionTest
   [junit4]   2> 966821 T3546 oas.SolrTestCaseJ4.initCore ####initCore
   [junit4]   2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-ExpressionTest-1389218852703
   [junit4]   2> 966822 T3546 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/'
   [junit4]   2> 966822 T3546 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 966822 T3546 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 966845 T3546 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 966872 T3546 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-basic.xml
   [junit4]   2> 966873 T3546 oass.IndexSchema.readSchema Reading Solr Schema from schema-analytics.xml
   [junit4]   2> 966876 T3546 oass.IndexSchema.readSchema [null] Schema name=schema-docValues
   [junit4]   2> 966888 T3546 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 966889 T3546 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 966889 T3546 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr
   [junit4]   2> 966889 T3546 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/'
   [junit4]   2> 966896 T3546 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 966897 T3546 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr
   [junit4]   2> 966897 T3546 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/'
   [junit4]   2> 966922 T3546 oasc.CoreContainer.<init> New CoreContainer 1339987834
   [junit4]   2> 966922 T3546 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/]
   [junit4]   2> 966922 T3546 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
   [junit4]   2> 966923 T3546 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 966923 T3546 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
   [junit4]   2> 966923 T3546 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 966923 T3546 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 966923 T3546 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 966923 T3546 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 966923 T3546 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 966924 T3546 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 966925 T3546 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 966925 T3546 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 966926 T3546 oasc.CoreContainer.load Host Name: 
   [junit4]   2> 966928 T3547 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1
   [junit4]   2> 966929 T3547 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/'
   [junit4]   2> 966929 T3547 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 966930 T3547 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 966961 T3547 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 966975 T3547 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-basic.xml
   [junit4]   2> 966975 T3547 oass.IndexSchema.readSchema Reading Solr Schema from schema-analytics.xml
   [junit4]   2> 966978 T3547 oass.IndexSchema.readSchema [collection1] Schema name=schema-docValues
   [junit4]   2> 966991 T3547 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 966992 T3547 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 966992 T3547 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-ExpressionTest-1389218852703/
   [junit4]   2> 966992 T3547 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
   [junit4]   2> 966992 T3547 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-ExpressionTest-1389218852703
   [junit4]   2> 966992 T3547 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-ExpressionTest-1389218852703/index/
   [junit4]   2> 966992 T3547 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-ExpressionTest-1389218852703/index' doesn't exist. Creating new index...
   [junit4]   2> 966993 T3547 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-ExpressionTest-1389218852703/index
   [junit4]   2> 966993 T3547 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=37, maxMergeAtOnceExplicit=25, maxMergedSegmentMB=54.4453125, floorSegmentMB=1.15625, forceMergeDeletesPctAllowed=1.0264031646500849, segmentsPerTier=48.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.2554379441229543
   [junit4]   2> 966994 T3547 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@751e441c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@24ed4e3e),segFN=segments_1,generation=1}
   [junit4]   2> 966994 T3547 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 966994 T3547 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 966995 T3547 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 966995 T3547 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 966997 T3547 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 966998 T3547 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 966998 T3547 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 966998 T3547 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=37, maxMergeAtOnceExplicit=34, maxMergedSegmentMB=32.5, floorSegmentMB=1.126953125, forceMergeDeletesPctAllowed=20.11358511914559, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 966998 T3547 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@751e441c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@24ed4e3e),segFN=segments_1,generation=1}
   [junit4]   2> 966999 T3547 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 966999 T3547 oass.SolrIndexSearcher.<init> Opening Searcher@2c470fb main
   [junit4]   2> 966999 T3547 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 966999 T3548 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2c470fb main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 967001 T3546 oas.SolrTestCaseJ4.initCore ####initCore end
   [junit4]   2> ASYNC  NEW_CORE C6892 name=collection1 org.apache.solr.core.SolrCore@b48db9c
   [junit4]   2> 967002 T3546 C6892 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 967003 T3546 C6892 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {deleteByQuery=*:*} 0 1
   [junit4]   2> ASYNC  NEW_CORE C6893 name=collection1 org.apache.solr.core.SolrCore@b48db9c
   [junit4]   2> 967016 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[10000]} 0 12
   [junit4]   2> 967020 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[10001]} 0 0
   [junit4]   2> 967023 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[10002]} 0 1
   [junit4]   2> 967025 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[10003]} 0 0
   [junit4]   2> 967028 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[10004]} 0 1
   [junit4]   2> 967030 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[10005]} 0 0
   [junit4]   2> 967033 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[10006]} 0 1
   [junit4]   2> 967035 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[10007]} 0 0
   [junit4]   2> 967038 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[10008]} 0 0
   [junit4]   2> 967040 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[10009]} 0 0
   [junit4]   2> 967043 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100010]} 0 0
   [junit4]   2> 967045 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100011]} 0 0
   [junit4]   2> 967048 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100012]} 0 0
   [junit4]   2> 967050 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100013]} 0 0
   [junit4]   2> 967053 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100014]} 0 0
   [junit4]   2> 967055 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100015]} 0 0
   [junit4]   2> 967058 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100016]} 0 1
   [junit4]   2> 967060 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100017]} 0 0
   [junit4]   2> 967062 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100018]} 0 0
   [junit4]   2> 967065 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100019]} 0 1
   [junit4]   2> 967067 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100020]} 0 0
   [junit4]   2> 967069 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100021]} 0 0
   [junit4]   2> 967071 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100022]} 0 0
   [junit4]   2> 967073 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100023]} 0 0
   [junit4]   2> 967076 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100024]} 0 1
   [junit4]   2> 967078 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100025]} 0 0
   [junit4]   2> 967080 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100026]} 0 0
   [junit4]   2> 967082 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100027]} 0 0
   [junit4]   2> 967084 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100028]} 0 0
   [junit4]   2> 967087 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100029]} 0 1
   [junit4]   2> 967089 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100030]} 0 0
   [junit4]   2> 967091 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100031]} 0 0
   [junit4]   2> 967093 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100032]} 0 0
   [junit4]   2> 967095 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100033]} 0 0
   [junit4]   2> 967098 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100034]} 0 1
   [junit4]   2> 967100 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100035]} 0 1
   [junit4]   2> 967102 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100036]} 0 0
   [junit4]   2> 967104 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100037]} 0 0
   [junit4]   2> 967106 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100038]} 0 0
   [junit4]   2> 967108 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100039]} 0 0
   [junit4]   2> 967111 T3546 C6893 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100040]} 0 1
   [junit4]   2> ASYNC  NEW_CORE C6894 name=collection1 org.apache.solr.core.SolrCore@b48db9c
   [junit4]   2> 967124 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100041]} 0 1
   [junit4]   2> 967126 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100042]} 0 0
   [junit4]   2> 967129 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100043]} 0 0
   [junit4]   2> 967132 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100044]} 0 1
   [junit4]   2> 967134 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100045]} 0 0
   [junit4]   2> 967137 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100046]} 0 1
   [junit4]   2> 967139 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100047]} 0 0
   [junit4]   2> 967142 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100048]} 0 1
   [junit4]   2> 967144 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100049]} 0 0
   [junit4]   2> 967147 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100050]} 0 0
   [junit4]   2> 967149 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100051]} 0 0
   [junit4]   2> 967152 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100052]} 0 0
   [junit4]   2> 967155 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100053]} 0 0
   [junit4]   2> 967158 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100054]} 0 1
   [junit4]   2> 967160 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100055]} 0 0
   [junit4]   2> 967163 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100056]} 0 1
   [junit4]   2> 967165 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100057]} 0 0
   [junit4]   2> 967168 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100058]} 0 1
   [junit4]   2> 967170 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100059]} 0 0
   [junit4]   2> 967173 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100060]} 0 1
   [junit4]   2> 967175 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100061]} 0 0
   [junit4]   2> 967178 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100062]} 0 1
   [junit4]   2> 967180 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100063]} 0 0
   [junit4]   2> 967183 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100064]} 0 1
   [junit4]   2> 967185 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100065]} 0 0
   [junit4]   2> 967188 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100066]} 0 1
   [junit4]   2> 967190 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100067]} 0 0
   [junit4]   2> 967193 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100068]} 0 1
   [junit4]   2> 967195 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100069]} 0 0
   [junit4]   2> 967198 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100070]} 0 1
   [junit4]   2> 967200 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100071]} 0 0
   [junit4]   2> 967202 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100072]} 0 0
   [junit4]   2> 967205 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100073]} 0 0
   [junit4]   2> 967207 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100074]} 0 0
   [junit4]   2> 967210 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100075]} 0 0
   [junit4]   2> 967212 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100076]} 0 0
   [junit4]   2> 967215 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100077]} 0 1
   [junit4]   2> 967217 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100078]} 0 0
   [junit4]   2> 967220 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100079]} 0 0
   [junit4]   2> 967222 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100080]} 0 0
   [junit4]   2> 967225 T3546 C6894 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100081]} 0 0
   [junit4]   2> ASYNC  NEW_CORE C6895 name=collection1 org.apache.solr.core.SolrCore@b48db9c
   [junit4]   2> 967238 T3546 C6895 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100082]} 0 0
   [junit4]   2> 967241 T3546 C6895 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100083]} 0 0
   [junit4]   2> 967244 T3546 C6895 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100084]} 0 0
   [junit4]   2> 967247 T3546 C6895 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100085]} 0 0
   [junit4]   2> 967250 T3546 C6895 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100086]} 0 1
   [junit4]   2> 967252 T3546 C6895 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100087]} 0 0
   [junit4]   2> 967255 T3546 C6895 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100088]} 0 0
   [junit4]   2> 967258 T3546 C6895 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100089]} 0 1
   [junit4]   2> 967260 T3546 C6895 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100090]} 0 0
   [junit4]   2> 967263 T3546 C6895 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100091]} 0 1
   [junit4]   2> 967265 T3546 C6895 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100092]} 0 0
   [junit4]   2> 967268 T3546 C6895 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100093]} 0 0
   [junit4]   2> 967271 T3546 C6895 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100094]} 0 1
   [junit4]   2> 967273 T3546 C6895 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100095]} 0 0
   [junit4]   2> 967276 T3546 C6895 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100096]} 0 0
   [junit4]   2> 967279 T3546 C6895 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100097]} 0 1
   [junit4]   2> 967282 T3546 C6895 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100098]} 0 1
   [junit4]   2> 967284 T3546 C6895 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[100099]} 0 0
   [junit4]   2> 967287 T3546 C6895 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 967292 T3546 C6895 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@751e441c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@24ed4e3e),segFN=segments_1,generation=1}
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@751e441c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@24ed4e3e),segFN=segments_2,generation=2}
   [junit4]   2> 967293 T3546 C6895 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
   [junit4]   2> 967294 T3546 C6895 oass.SolrIndexSearcher.<init> Opening Searcher@10e9c06d main
   [junit4]   2> 967294 T3546 C6895 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 967294 T3548 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@10e9c06d main{StandardDirectoryReader(segments_2:4:nrt _0(5.0):c100)}
   [junit4]   2> 967294 T3546 C6895 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {commit=} 0 7
   [junit4]   2> 967309 T3546 C6895 oasc.SolrCore.execute [collection1] webapp=null path=null params={o.dmr.s.median=median(date_dtd)&stats=true&o.avr.s.s=abs(neg(sum(int_id)))&o.mr.s.mean=mean(int_id)&o.cr.s.ccmax=concat(const_str(this+is+the+second),max(string_sd))&olap=true&o.cr.s.ccmin=concat(const_str(this+is+the+first),min(string_sd))&o.dr.s.mean=mean(int_id)&rows=0&o.cnr.s.c8=const_num(8)&o.dr.s.mc=div(mean(int_id),count(long_ld))&o.rr.s.rmax=rev(max(string_sd))&o.csr.s.cs2=const_str(this+is+the+second)&o.nr.s.c=neg(count(long_ld))&o.nr.s.count=count(long_ld)&o.cdr.s.cd1=const_date(1800-12-31T23:59:59Z)&o.csr.s.cs1=const_str(this+is+the+first)&o.dr.s.count=count(long_ld)&o.dmr.s.max=max(date_dtd)&o.mr.s.count=count(long_ld)&o.cr.s.max=max(string_sd)&o.dr.s.su=div(sum(int_id),unique(long_ld))&o.rr.s.max=max(string_sd)&o.mr.s.sum=sum(int_id)&o.ar.s.median=median(int_id)&o.ar.s.su=add(sum(int_id),unique(long_ld))&o.cdr.s.cd2=const_date(1804-06-30T23:59:59Z)&o.mr.s.mcm=mult(mean(int_id),count(long_ld),median(int_id))&o.mr.s.unique=unique(long_ld)&o.pr.s.su=pow(sum(int_id),unique(long_ld))&o.ar.s.mcm=add(mean(int_id),count(long_ld),median(int_id))&o.pr.s.mean=mean(int_id)&o.cr.s.min=min(string_sd)&o.cr.s.csmax=const_str(this+is+the+second)&o.cnr.s.c10=const_num(10)&o.dr.s.unique=unique(long_ld)&o.pr.s.sum=sum(int_id)&o.rr.s.rmin=rev(min(string_sd))&o.dr.s.sum=sum(int_id)&o.nr.s.sum=sum(int_id)&o.dmr.s.cma=const_str(%2B2MONTHS)&o.mr.s.su=mult(sum(int_id),unique(long_ld))&o.cdr.s.cs2=const_str(1804-06-30T23:59:59Z)&o.rr.s.min=min(string_sd)&o.ar.s.count=count(long_ld)&o.dmr.s.dmme=date_math(median(date_dtd),const_str(%2B2YEARS))&o.mr.s.median=median(int_id)&o.avr.s.count=count(long_ld)&o.dmr.s.dmma=date_math(max(date_dtd),const_str(%2B2MONTHS))&o.csr.s.cs3=const_str(this+is+the+third)&o.dmr.s.cme=const_str(%2B2YEARS)&o.cdr.s.cs1=const_str(1800-12-31T23:59:59Z)&o.ar.s.unique=unique(long_ld)&o.cr.s.csmin=const_str(this+is+the+first)&o.ar.s.sum=sum(int_id)&o.pr.s.count=count(long_ld)&indent=true&o.avr.s.c=abs(neg(count(long_ld)))&o.pr.s.unique=unique(long_ld)&o.avr.s.sum=sum(int_id)&o.pr.s.mc=pow(mean(int_id),count(long_ld))&o.nr.s.s=neg(sum(int_id))&o.ar.s.mean=mean(int_id)&q=*:*} hits=100 status=0 QTime=12 
   [junit4]   2> 967315 T3546 oas.SolrTestCaseJ4.setUp ###Starting negateTest
   [junit4]   2> 967317 T3546 oas.SolrTestCaseJ4.tearDown ###Ending negateTest
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ExpressionTest -Dtests.method=negateTest -Dtests.seed=3880E4F26B38C358 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ca -Dtests.timezone=Europe/Stockholm -Dtests.file.encoding=US-ASCII
   [junit4] FAILURE 0.02s J1 | ExpressionTest.negateTest <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: Caught exception in getStatResult, xPath = /response/lst[@name='stats']/lst[@name='nr']/double[@name='sum'] 
   [junit4]    > raw data: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]    > <response>
   [junit4]    > <lst name="responseHeader">
   [junit4]    >   <int name="status">0</int>
   [junit4]    >   <int name="QTime">12</int>
   [junit4]    > </lst>
   [junit4]    > <result name="response" numFound="100" start="0">
   [junit4]    > </result>
   [junit4]    > <lst name="stats">
   [junit4]    >   <lst name="stats_fields"/>
   [junit4]    > </lst>
   [junit4]    > <lst name="stats">
   [junit4]    >   <lst name="rr">
   [junit4]    >     <str name="max">str9</str>
   [junit4]    >     <str name="min">str0</str>
   [junit4]    >     <str name="rmax">9rts</str>
   [junit4]    >     <str name="rmin">0rts</str>
   [junit4]    >   </lst>
   [junit4]    >   <lst name="dmr">
   [junit4]    >     <str name="cma">+2MONTHS</str>
   [junit4]    >     <str name="cme">+2YEARS</str>
   [junit4]    >     <date name="dmma">1812-02-29T23:59:59Z</date>
   [junit4]    >     <date name="dmme">1807-12-31T23:59:59Z</date>
   [junit4]    >     <date name="max">1811-12-31T23:59:59Z</date>
   [junit4]    >     <date name="median">1805-12-31T23:59:59Z</date>
   [junit4]    >   </lst>
   [junit4]    >   <lst name="cr">
   [junit4]    >     <str name="ccmax">this is the secondstr9</str>
   [junit4]    >     <str name="ccmin">this is the firststr0</str>
   [junit4]    >     <str name="csmax">this is the second</str>
   [junit4]    >     <str name="csmin">this is the first</str>
   [junit4]    >     <str name="max">str9</str>
   [junit4]    >     <str name="min">str0</str>
   [junit4]    >   </lst>
   [junit4]    >   <lst name="mr">
   [junit4]    >     <long name="count">100</long>
   [junit4]    >     <null name="mcm"/>
   [junit4]    >     <null name="mean"/>
   [junit4]    >     <null name="median"/>
   [junit4]    >     <null name="su"/>
   [junit4]    >     <null name="sum"/>
   [junit4]    >     <null name="unique"/>
   [junit4]    >   </lst>
   [junit4]    >   <lst name="nr">
   [junit4]    >     <double name="c">-100.0</double>
   [junit4]    >     <long name="count">100</long>
   [junit4]    >     <null name="s"/>
   [junit4]    >     <null name="sum"/>
   [junit4]    >   </lst>
   [junit4]    >   <lst name="avr">
   [junit4]    >     <double name="c">100.0</double>
   [junit4]    >     <long name="count">100</long>
   [junit4]    >     <null name="s"/>
   [junit4]    >     <null name="sum"/>
   [junit4]    >   </lst>
   [junit4]    >   <lst name="cnr">
   [junit4]    >     <double name="c10">10.0</double>
   [junit4]    >     <double name="c8">8.0</double>
   [junit4]    >   </lst>
   [junit4]    >   <lst name="pr">
   [junit4]    >     <long name="count">100</long>
   [junit4]    >     <null name="mc"/>
   [junit4]    >     <null name="mean"/>
   [junit4]    >     <null name="su"/>
   [junit4]    >     <null name="sum"/>
   [junit4]    >     <null name="unique"/>
   [junit4]    >   </lst>
   [junit4]    >   <lst name="cdr">
   [junit4]    >     <date name="cd1">1800-12-31T23:59:59Z</date>
   [junit4]    >     <date name="cd2">1804-06-30T23:59:59Z</date>
   [junit4]    >     <str name="cs1">1800-12-31T23:59:59Z</str>
   [junit4]    >     <str name="cs2">1804-06-30T23:59:59Z</str>
   [junit4]    >   </lst>
   [junit4]    >   <lst name="csr">
   [junit4]    >     <str name="cs1">this is the first</str>
   [junit4]    >     <str name="cs2">this is the second</str>
   [junit4]    >     <str name="cs3">this is the third</str>
   [junit4]    >   </lst>
   [junit4]    >   <lst name="dr">
   [junit4]    >     <long name="count">100</long>
   [junit4]    >     <null name="mc"/>
   [junit4]    >     <null name="mean"/>
   [junit4]    >     <null name="su"/>
   [junit4]    >     <null name="sum"/>
   [junit4]    >     <null name="unique"/>
   [junit4]    >   </lst>
   [junit4]    >   <lst name="ar">
   [junit4]    >     <long name="count">100</long>
   [junit4]    >     <null name="mcm"/>
   [junit4]    >     <null name="mean"/>
   [junit4]    >

[...truncated too long message...]

1-12-31T23:59:59Z</date>
   [junit4]    >     <date name="median">1805-12-31T23:59:59Z</date>
   [junit4]    >   </lst>
   [junit4]    >   <lst name="cr">
   [junit4]    >     <str name="ccmax">this is the secondstr9</str>
   [junit4]    >     <str name="ccmin">this is the firststr0</str>
   [junit4]    >     <str name="csmax">this is the second</str>
   [junit4]    >     <str name="csmin">this is the first</str>
   [junit4]    >     <str name="max">str9</str>
   [junit4]    >     <str name="min">str0</str>
   [junit4]    >   </lst>
   [junit4]    >   <lst name="mr">
   [junit4]    >     <long name="count">100</long>
   [junit4]    >     <null name="mcm"/>
   [junit4]    >     <null name="mean"/>
   [junit4]    >     <null name="median"/>
   [junit4]    >     <null name="su"/>
   [junit4]    >     <null name="sum"/>
   [junit4]    >     <null name="unique"/>
   [junit4]    >   </lst>
   [junit4]    >   <lst name="nr">
   [junit4]    >     <double name="c">-100.0</double>
   [junit4]    >     <long name="count">100</long>
   [junit4]    >     <null name="s"/>
   [junit4]    >     <null name="sum"/>
   [junit4]    >   </lst>
   [junit4]    >   <lst name="avr">
   [junit4]    >     <double name="c">100.0</double>
   [junit4]    >     <long name="count">100</long>
   [junit4]    >     <null name="s"/>
   [junit4]    >     <null name="sum"/>
   [junit4]    >   </lst>
   [junit4]    >   <lst name="cnr">
   [junit4]    >     <double name="c10">10.0</double>
   [junit4]    >     <double name="c8">8.0</double>
   [junit4]    >   </lst>
   [junit4]    >   <lst name="pr">
   [junit4]    >     <long name="count">100</long>
   [junit4]    >     <null name="mc"/>
   [junit4]    >     <null name="mean"/>
   [junit4]    >     <null name="su"/>
   [junit4]    >     <null name="sum"/>
   [junit4]    >     <null name="unique"/>
   [junit4]    >   </lst>
   [junit4]    >   <lst name="cdr">
   [junit4]    >     <date name="cd1">1800-12-31T23:59:59Z</date>
   [junit4]    >     <date name="cd2">1804-06-30T23:59:59Z</date>
   [junit4]    >     <str name="cs1">1800-12-31T23:59:59Z</str>
   [junit4]    >     <str name="cs2">1804-06-30T23:59:59Z</str>
   [junit4]    >   </lst>
   [junit4]    >   <lst name="csr">
   [junit4]    >     <str name="cs1">this is the first</str>
   [junit4]    >     <str name="cs2">this is the second</str>
   [junit4]    >     <str name="cs3">this is the third</str>
   [junit4]    >   </lst>
   [junit4]    >   <lst name="dr">
   [junit4]    >     <long name="count">100</long>
   [junit4]    >     <null name="mc"/>
   [junit4]    >     <null name="mean"/>
   [junit4]    >     <null name="su"/>
   [junit4]    >     <null name="sum"/>
   [junit4]    >     <null name="unique"/>
   [junit4]    >   </lst>
   [junit4]    >   <lst name="ar">
   [junit4]    >     <long name="count">100</long>
   [junit4]    >     <null name="mcm"/>
   [junit4]    >     <null name="mean"/>
   [junit4]    >     <null name="median"/>
   [junit4]    >     <null name="su"/>
   [junit4]    >     <null name="sum"/>
   [junit4]    >     <null name="unique"/>
   [junit4]    >   </lst>
   [junit4]    > </lst>
   [junit4]    > </response>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([3880E4F26B38C358:608D0F044FFAA18E]:0)
   [junit4]    > 	at org.apache.solr.analytics.AbstractAnalyticsStatsTest.getStatResult(AbstractAnalyticsStatsTest.java:113)
   [junit4]    > 	at org.apache.solr.analytics.expression.ExpressionTest.divideTest(ExpressionTest.java:106)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 967407 T3546 oas.SolrTestCaseJ4.setUp ###Starting constantStringTest
   [junit4]   2> 967409 T3546 oas.SolrTestCaseJ4.tearDown ###Ending constantStringTest
   [junit4]   2> 967415 T3546 oas.SolrTestCaseJ4.setUp ###Starting concatenateTest
   [junit4]   2> 967418 T3546 oas.SolrTestCaseJ4.tearDown ###Ending concatenateTest
   [junit4]   2> 967419 T3546 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> 967419 T3546 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1339987834
   [junit4]   2> 967420 T3546 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@b48db9c
   [junit4]   2> 967420 T3546 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=100,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
   [junit4]   2> 967420 T3546 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 967420 T3546 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 967420 T3546 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 967421 T3546 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 967421 T3546 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 967421 T3546 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-ExpressionTest-1389218852703/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-ExpressionTest-1389218852703/index;done=false>>]
   [junit4]   2> 967421 T3546 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-ExpressionTest-1389218852703/index
   [junit4]   2> 967421 T3546 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-ExpressionTest-1389218852703 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-ExpressionTest-1389218852703;done=false>>]
   [junit4]   2> 967422 T3546 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-ExpressionTest-1389218852703
   [junit4]   2> NOTE: test params are: codec=Lucene45, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=ca, timezone=Europe/Stockholm
   [junit4]   2> NOTE: Linux 3.8.0-35-generic amd64/Oracle Corporation 1.7.0_60-ea (64-bit)/cpus=8,threads=1,free=199653152,total=357732352
   [junit4]   2> NOTE: All tests run in this JVM: [TermVectorComponentDistributedTest, ConvertedLegacyTest, TestSerializedLuceneMatchVersion, TestDistributedSearch, DateFieldTest, TestRandomMergePolicy, SolrIndexConfigTest, RequestHandlersTest, TestFastOutputStream, TestIndexingPerformance, SpellingQueryConverterTest, LeaderElectionIntegrationTest, TestCursorMarkWithoutUniqueKey, OpenCloseCoreStressTest, TestRTGBase, TestCloudManagedSchema, TestAnalyzedSuggestions, XmlUpdateRequestHandlerTest, TestNumberUtils, SolrRequestParserTest, TestRandomFaceting, FullSolrCloudDistribCmdsTest, TestSchemaResource, TestPhraseSuggestions, UniqFieldsUpdateProcessorFactoryTest, TestOmitPositions, DirectSolrConnectionTest, SpellCheckComponentTest, TestComponentsName, RequiredFieldsTest, TestPartialUpdateDeduplication, TestAtomicUpdateErrorCases, TestStressReorder, TestStressUserVersions, QueryFacetTest, TestBinaryField, PreAnalyzedUpdateProcessorTest, TestXIncludeConfig, TestFieldResource, DefaultValueUpdateProcessorTest, ZkSolrClientTest, AlternateDirectoryTest, CachingDirectoryFactoryTest, TestBinaryResponseWriter, TestFieldCollectionResource, DistribCursorPagingTest, TestStressLucene, TestFuzzyAnalyzedSuggestions, TestSimpleQParserPlugin, DisMaxRequestHandlerTest, SignatureUpdateProcessorFactoryTest, CollectionsAPIDistributedZkTest, TestSolrDeletionPolicy2, SuggesterTSTTest, TestSolrQueryParserDefaultOperatorResource, TestManagedSchemaFieldResource, TestDistributedGrouping, TestFoldingMultitermQuery, OpenExchangeRatesOrgProviderTest, TestStressRecovery, NumericFieldsTest, TestSweetSpotSimilarityFactory, ZkNodePropsTest, TestReload, RangeFacetTest, TestUniqueKeyFieldResource, TestLazyCores, BadIndexSchemaTest, ChaosMonkeySafeLeaderTest, StandardRequestHandlerTest, TestFileDictionaryLookup, BlockDirectoryTest, TestFiltering, TestQuerySenderListener, TestRecovery, FileUtilsTest, CurrencyFieldXmlFileTest, FieldFacetExtrasTest, DeleteReplicaTest, ExpressionTest]
   [junit4] Completed on J1 in 0.62s, 12 tests, 6 failures <<< FAILURES!

[...truncated 559 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:453: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:433: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:491: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1304: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:937: There were test failures: 362 suites, 1586 tests, 6 failures, 34 ignored (6 assumptions)

Total time: 41 minutes 41 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0_60-ea-b02 -XX:+UseCompressedOops -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.7.0_45) - Build # 9017 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/9017/
Java: 64bit/jdk1.7.0_45 -XX:+UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch

Error Message:


Stack Trace:
java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([DBFAFC4DB6DE3BA0:5A1C7255C1815B9C]:0)
	at org.junit.Assert.fail(Assert.java:92)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertFalse(Assert.java:68)
	at org.junit.Assert.assertFalse(Assert.java:79)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.deleteCollectionWithDownNodes(CollectionsAPIDistributedZkTest.java:390)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:203)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:867)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:744)




Build Log:
[...truncated 10216 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> 426343 T1995 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /io/
   [junit4]   2> 426350 T1995 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-CollectionsAPIDistributedZkTest-1389212301833
   [junit4]   2> 426351 T1995 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 426352 T1996 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 426452 T1995 oasc.ZkTestServer.run start zk server on port:43218
   [junit4]   2> 426453 T1995 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 426468 T2002 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@69ef552c name:ZooKeeperConnection Watcher:127.0.0.1:43218 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 426468 T1995 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 426468 T1995 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 426477 T1995 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 426479 T2004 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5a7ed5cb name:ZooKeeperConnection Watcher:127.0.0.1:43218/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 426479 T1995 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 426480 T1995 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 426486 T1995 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 426491 T1995 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 426496 T1995 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 426502 T1995 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 426502 T1995 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 426511 T1995 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 426512 T1995 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 426519 T1995 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 426520 T1995 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 426525 T1995 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 426526 T1995 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 426532 T1995 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 426533 T1995 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 426536 T1995 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 426536 T1995 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 426539 T1995 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 426539 T1995 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 426543 T1995 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 426544 T1995 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 426549 T1995 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 426549 T1995 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 426554 T1995 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 426554 T1995 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 426557 T1995 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 426557 T1995 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> Using old style solr.xml
   [junit4]   2> 426561 T1995 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 426562 T2006 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@598dcc2a name:ZooKeeperConnection Watcher:127.0.0.1:43218 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 426562 T1995 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 426563 T1995 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 426565 T1995 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 426566 T2008 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d4b8e6d name:ZooKeeperConnection Watcher:127.0.0.1:43218/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 426567 T1995 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 426567 T1995 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 426568 T1995 oascc.SolrZkClient.makePath makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 426571 T1995 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 426571 T1995 oascc.SolrZkClient.makePath makePath: /configs/conf2/schema.xml
   [junit4]   2> 426575 T1995 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 426575 T1995 oascc.SolrZkClient.makePath makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 426579 T1995 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 426579 T1995 oascc.SolrZkClient.makePath makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 426582 T1995 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 426582 T1995 oascc.SolrZkClient.makePath makePath: /configs/conf2/protwords.txt
   [junit4]   2> 426585 T1995 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 426585 T1995 oascc.SolrZkClient.makePath makePath: /configs/conf2/currency.xml
   [junit4]   2> 426588 T1995 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 426588 T1995 oascc.SolrZkClient.makePath makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 426590 T1995 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 426591 T1995 oascc.SolrZkClient.makePath makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 426593 T1995 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 426593 T1995 oascc.SolrZkClient.makePath makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 426596 T1995 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 426596 T1995 oascc.SolrZkClient.makePath makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 426598 T1995 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 426599 T1995 oascc.SolrZkClient.makePath makePath: /configs/conf2/elevate.xml
   [junit4]   2> 426668 T1995 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 426687 T1995 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:36776
   [junit4]   2> 426688 T1995 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 426689 T1995 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 426689 T1995 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1389212302085
   [junit4]   2> 426689 T1995 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1389212302085/'
   [junit4]   2> 426712 T1995 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1389212302085/solr.xml
   [junit4]   2> 426744 T1995 oasc.CoreContainer.<init> New CoreContainer 1059613292
   [junit4]   2> 426744 T1995 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1389212302085/]
   [junit4]   2> 426745 T1995 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 426746 T1995 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 426746 T1995 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 426746 T1995 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 426746 T1995 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 426746 T1995 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 426747 T1995 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 426747 T1995 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 426747 T1995 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 426749 T1995 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 426749 T1995 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 426749 T1995 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 426750 T1995 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:43218/solr
   [junit4]   2> 426750 T1995 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 426751 T1995 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 426753 T2019 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7662f8d8 name:ZooKeeperConnection Watcher:127.0.0.1:43218 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 426753 T1995 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 426757 T1995 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 426759 T2021 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3d0a3354 name:ZooKeeperConnection Watcher:127.0.0.1:43218/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 426760 T1995 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 426762 T1995 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 426769 T1995 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 426775 T1995 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 426780 T1995 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36776_io
   [junit4]   2> 426781 T1995 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:36776_io
   [junit4]   2> 426787 T1995 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 426791 T1995 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 426800 T1995 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 426804 T1995 oasc.Overseer.start Overseer (id=91043417413189637-127.0.0.1:36776_io-n_0000000000) starting
   [junit4]   2> 426809 T1995 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 426816 T2023 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 426817 T1995 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 426822 T1995 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 426825 T1995 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 426830 T2022 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 426834 T1995 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 426835 T1995 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 426836 T1995 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 426838 T2026 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@526fd458 name:ZooKeeperConnection Watcher:127.0.0.1:43218/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 426839 T1995 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 426841 T1995 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 426846 T1995 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 426919 T1995 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 426922 T1995 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:42305
   [junit4]   2> 426922 T1995 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 426923 T1995 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 426923 T1995 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1389212302329
   [junit4]   2> 426924 T1995 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1389212302329/'
   [junit4]   2> 426940 T1995 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1389212302329/solr.xml
   [junit4]   2> 426971 T1995 oasc.CoreContainer.<init> New CoreContainer 1195188394
   [junit4]   2> 426971 T1995 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1389212302329/]
   [junit4]   2> 426972 T1995 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 426972 T1995 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 426972 T1995 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 426973 T1995 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 426973 T1995 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 426973 T1995 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 426973 T1995 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 426973 T1995 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 426974 T1995 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 426976 T1995 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 426976 T1995 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 426976 T1995 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 426976 T1995 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:43218/solr
   [junit4]   2> 426977 T1995 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 426977 T1995 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 426980 T2037 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7e2b9d6f name:ZooKeeperConnection Watcher:127.0.0.1:43218 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 426980 T1995 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 426983 T1995 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 426985 T2039 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17639837 name:ZooKeeperConnection Watcher:127.0.0.1:43218/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 426985 T1995 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 426991 T1995 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 427996 T1995 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42305_io
   [junit4]   2> 427999 T1995 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:42305_io
   [junit4]   2> 428005 T2021 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 428005 T2026 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 428006 T2039 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 428016 T1995 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 428016 T1995 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 428088 T1995 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 428091 T1995 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:40584
   [junit4]   2> 428092 T1995 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 428093 T1995 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 428093 T1995 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1389212303500
   [junit4]   2> 428093 T1995 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1389212303500/'
   [junit4]   2> 428115 T1995 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1389212303500/solr.xml
   [junit4]   2> 428146 T1995 oasc.CoreContainer.<init> New CoreContainer 655841928
   [junit4]   2> 428146 T1995 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1389212303500/]
   [junit4]   2> 428147 T1995 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 428148 T1995 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 428148 T1995 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 428148 T1995 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 428148 T1995 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 428148 T1995 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 428149 T1995 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 428149 T1995 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 428149 T1995 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 428151 T1995 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 428151 T1995 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 428152 T1995 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 428152 T1995 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:43218/solr
   [junit4]   2> 428152 T1995 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 428153 T1995 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 428155 T2051 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6e38aac4 name:ZooKeeperConnection Watcher:127.0.0.1:43218 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 428156 T1995 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 428160 T1995 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 428162 T2053 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7b1a284e name:ZooKeeperConnection Watcher:127.0.0.1:43218/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 428162 T1995 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 428169 T1995 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 429176 T1995 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40584_io
   [junit4]   2> 429178 T1995 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:40584_io
   [junit4]   2> 429184 T2039 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 429184 T2026 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 429184 T2021 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 429184 T2053 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 429195 T1995 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 429195 T1995 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 429267 T1995 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 429270 T1995 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:40932
   [junit4]   2> 429270 T1995 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 429270 T1995 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 429271 T1995 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1389212304679
   [junit4]   2> 429271 T1995 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1389212304679/'
   [junit4]   2> 429282 T1995 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1389212304679/solr.xml
   [junit4]   2> 429331 T1995 oasc.CoreContainer.<init> New CoreContainer 1596999934
   [junit4]   2> 429331 T1995 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1389212304679/]
   [junit4]   2> 429332 T1995 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 429332 T1995 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 429332 T1995 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 429332 T1995 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 429333 T1995 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 429333 T1995 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 429333 T1995 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 429333 T1995 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 429333 T1995 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 429336 T1995 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 429336 T1995 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 429336 T1995 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 429337 T1995 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:43218/solr
   [junit4]   2> 429337 T1995 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 429338 T1995 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 429340 T2065 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5b084c84 name:ZooKeeperConnection Watcher:127.0.0.1:43218 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 429340 T1995 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 429343 T1995 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 429344 T2067 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@105ac953 name:ZooKeeperConnection Watcher:127.0.0.1:43218/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 429345 T1995 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 429354 T1995 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 430359 T1995 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40932_io
   [junit4]   2> 430361 T1995 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:40932_io
   [junit4]   2> 430367 T2026 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 430367 T2039 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 430367 T2067 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 430367 T2021 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 430368 T2053 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 430377 T1995 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 430378 T1995 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 430450 T1995 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 430453 T1995 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:33557
   [junit4]   2> 430453 T1995 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 430453 T1995 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 430454 T1995 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1389212305861
   [junit4]   2> 430454 T1995 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1389212305861/'
   [junit4]   2> 430475 T1995 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1389212305861/solr.xml
   [junit4]   2> 430506 T1995 oasc.CoreContainer.<init> New CoreContainer 1305185281
   [junit4]   2> 430506 T1995 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1389212305861/]
   [junit4]   2> 430507 T1995 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 430507 T1995 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 430508 T1995 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 430508 T1995 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 430508 T1995 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 430508 T1995 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 430509 T1995 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 430509 T1995 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 430509 T1995 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 430511 T1995 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 430511 T1995 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 430511 T1995 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 430512 T1995 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:43218/solr
   [junit4]   2> 430512 T1995 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 430513 T1995 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 430515 T2079 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6e54bd8d name:ZooKeeperConnection Watcher:127.0.0.1:43218 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 430515 T1995 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 430519 T1995 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 430521 T2081 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4c40bdf name:ZooKeeperConnection Watcher:127.0.0.1:43218/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 430521 T1995 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 430528 T1995 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 431535 T1995 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33557_io
   [junit4]   2> 431538 T1995 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:33557_io
   [junit4]   2> 431544 T2053 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 431544 T2021 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 431544 T2039 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 431544 T2026 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 431544 T2067 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 431544 T2081 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 431554 T1995 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 431555 T1995 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 431570 T2034 oasha.CollectionsHandler.handleCreateAction Creating Collection : collection.configName=conf1&router.field=myOwnField&name=solrj_collection&version=2&action=CREATE&wt=javabin&numShards=2&replicationFactor=2&router.name=compositeId
   [junit4]   2> 431572 T2021 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 431575 T2023 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
   [junit4]   2> 	  "operation":"createcollection",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"solrj_collection",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "collection.configName":"conf1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "router.field":"myOwnField",
   [junit4]   2> 	  "router.name":"compositeId"}
   [junit4]   2> 431575 T2023 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : createcollection , {
   [junit4]   2> 	  "operation":"createcollection",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"solrj_collection",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "collection.configName":"conf1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "router.field":"myOwnField",
   [junit4]   2> 	  "router.name":"compositeId"}
   [junit4]   2> 431575 T2023 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/solrj_collection 
   [junit4]   2> 431576 T2023 oascc.SolrZkClient.makePath makePath: /collections/solrj_collection
   [junit4]   2> 431582 T2021 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 431584 T2022 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 431585 T2022 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: solrj_collection
   [junit4]   2> 431586 T2022 oasc.Overseer$ClusterStateUpdater.createCollection Create collection solrj_collection with shards [shard1, shard2]
   [junit4]   2> 431590 T2021 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 431592 T2026 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 431592 T2081 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 431592 T2021 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 431592 T2053 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 431592 T2067 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 431592 T2039 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 431682 T2023 oasc.OverseerCollectionProcessor.createCollection going to create cores replicas shardNames [shard1, shard2] , repFactor : 2
   [junit4]   2> 431683 T2023 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard1_replica1 as part of slice shard1 of collection solrj_collection on 127.0.0.1:40932_io
   [junit4]   2> 431683 T2023 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard1_replica2 as part of slice shard1 of collection solrj_collection on 127.0.0.1:36776_io
   [junit4]   2> 431685 T2023 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard2_replica1 as part of slice shard2 of collection solrj_collection on 127.0.0.1:42305_io
   [junit4]   2> 431685 T2023 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard2_replica2 as part of slice shard2 of collection solrj_collection on 127.0.0.1:40584_io
   [junit4]   2> 431687 T2060 oasc.ZkController.publish publishing core=solrj_collection_shard1_replica1 state=down
   [junit4]   2> 431687 T2017 oasc.ZkController.publish publishing core=solrj_collection_shard1_replica2 state=down
   [junit4]   2> 431688 T2035 oasc.ZkController.publish publishing core=solrj_collection_shard2_replica1 state=down
   [junit4]   2> 431689 T2060 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 431690 T2017 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 431690 T2049 oasc.ZkController.publish publishing core=solrj_collection_shard2_replica2 state=down
   [junit4]   2> 431691 T2035 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 431689 T2021 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 431692 T2049 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 431694 T2022 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 431695 T2022 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:40932/io",
   [junit4]   2> 	  "core":"solrj_collection_shard1_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:40932_io",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"solrj_collection",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 431700 T2021 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 431701 T2022 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:36776/io",
   [junit4]   2> 	  "core":"solrj_collection_shard1_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:36776_io",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"solrj_collection",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 431706 T2021 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 431708 T2022 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:42305/io",
   [junit4]   2> 	  "core":"solrj_collection_shard2_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:42305_io",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"solrj_collection",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 431711 T2021 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 431713 T2022 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:40584/io",
   [junit4]   2> 	  "core":"solrj_collection_shard2_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:40584_io",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"solrj_collection",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 431720 T2021 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 431824 T2021 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 431825 T2026 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 431825 T2039 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 431825 T2053 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 431825 T2081 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 431825 T2067 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 432690 T2060 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard1_replica1
   [junit4]   2> 432691 T2017 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard1_replica2
   [junit4]   2> 432691 T2060 oasc.SolrXMLCoresLocator.doPersist Persisted core descriptions to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1389212304679/solr.xml
   [junit4]   2> 432691 T2017 oasc.SolrXMLCoresLocator.doPersist Persisted core descriptions to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1389212302085/solr.xml
   [junit4]   2> 432692 T2017 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard1_replica2' using instanceDir: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1389212302085/solrj_collection_shard1_replica2
   [junit4]   2> 432692 T2017 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
   [junit4]   2> 432692 T2035 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard2_replica1
   [junit4]   2> 432691 T2060 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard1_replica1' using instanceDir: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1389212304679/solrj_collection_shard1_replica1
   [junit4]   2> 432694 T2060 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
   [junit4]   2> 432693 T2049 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard2_replica2
   [junit4]   2> 432694 T2017 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 432695 T2060 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 432695 T2060 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
   [junit4]   2> 432694 T2035 oasc.SolrXMLCoresLocator.doPersist Persisted core descriptions to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1389212302329/solr.xml
   [junit4]   2> 432695 T2017 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
   [junit4]   2> 432695 T2049 oasc.SolrXMLCoresLocator.doPersist Persisted core descriptions to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1389212303500/solr.xml
   [junit4]   2> 432696 T2035 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard2_replica1' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1389212302329/solrj_collection_shard2_replica1
   [junit4]   2> 432698 T2035 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
   [junit4]   2> 432697 T2049 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard2_replica2' using instanceDir: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1389212303500/solrj_collection_shard2_replica2
   [junit4]   2> 432699 T2049 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
   [junit4]   2> 432698 T2060 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1389212304679/solrj_collection_shard1_replica1/'
   [junit4]   2> 432699 T2035 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 432700 T2049 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 432700 T2049 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
   [junit4]   2> 432699 T2017 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1389212302085/solrj_collection_shard1_replica2/'
   [junit4]   2> 432700 T2035 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
   [junit4]   2> 432702 T2049 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1389212303500/solrj_collection_shard2_replica2/'
   [junit4]   2> 432704 T2035 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1389212302329/solrj_collection_shard2_replica1/'
   [junit4]   2> 432751 T2060 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 432756 T2035 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 432763 T2049 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 432773 T2017 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 432785 T2060 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 432787 T2060 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 432790 T2035 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 432792 T2035 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 432794 T2060 oass.IndexSchema.readSchema [solrj_collection_shard1_replica1] Schema name=test
   [junit4]   2> 432798 T2035 oass.IndexSchema.readSchema [solrj_collection_shard2_replica1] Schema name=test
   [junit4]   2> 432811 T2049 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 432813 T2049 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 432818 T2017 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 432819 T2049 oass.IndexSchema.readSchema [solrj_collection_shard2_replica2] Schema name=test
   [junit4]   2> 432820 T2017 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 432826 T2017 oass.IndexSchema.readSchema [solrj_collection_shard1_replica2] Schema name=test
   [junit4]   2> 433169 T2060 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 433175 T2060 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 433177 T2060 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 433196 T2035 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 433201 T2035 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 433204 T2035 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 433205 T2060 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 433207 T2017 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 433209 T2060 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 433213 T2017 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 433213 T2060 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 433215 T2060 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 433215 T2060 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 433215 T2060 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 433216 T2017 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 433217 T2060 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 433217 T2035 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 433217 T2060 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 433218 T2060 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 433218 T2060 oasc.SolrCore.<init> [solrj_collection_shard1_replica1] Opening new SolrCore at ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1389212304679/solrj_collection_shard1_replica1/, dataDir=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1389212304679/solrj_collection_shard1_replica1/data/
   [junit4]   2> 433218 T2060 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@46bcb6d2
   [junit4]   2> 433219 T2060 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1389212304679/solrj_collection_shard1_replica1/data
   [junit4]   2> 433220 T2060 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1389212304679/solrj_collection_shard1_replica1/data/index/
   [junit4]   2> 433220 T2060 oasc.SolrCore.initIndex WARN [solrj_collection_shard1_replica1] Solr index directory './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1389212304679/solrj_collection_shard1_replica1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 433220 T2035 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 433221 T2060 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1389212304679/solrj_collection_shard1_replica1/data/index
   [junit4]   2> 433221 T2060 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=38, maxMergeAtOnceExplicit=42, maxMergedSegmentMB=96.64453125, floorSegmentMB=2.1533203125, forceMergeDeletesPctAllowed=3.9670214619663904, segmentsPerTier=37.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 433223 T2060 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/index3619636208tmp lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@64c837c5),segFN=segments_1,generation=1}
   [junit4]   2> 433223 T2060 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 433224 T2035 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 433225 T2035 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 433225 T2035 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 433225 T2035 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 433226 T2035 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 433227 T2035 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 433227 T2035 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 433227 T2035 oasc.SolrCore.<init> [solrj_collection_shard2_replica1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1389212302329/solrj_collection_shard2_replica1/, dataDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1389212302329/solrj_collection_shard2_replica1/data/
   [junit4]   2> 433228 T2035 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@46bcb6d2
   [junit4]   2> 433228 T2017 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 433228 T2035 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1389212302329/solrj_collection_shard2_replica1/data
   [junit4]   2> 433229 T2035 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1389212302329/solrj_collection_shard2_replica1/data/index/
   [junit4]   2> 433229 T2060 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 433229 T2035 oasc.SolrCore.initIndex WARN [solrj_collection_shard2_replica1] Solr index directory '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1389212302329/solrj_collection_shard2_replica1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 433229 T2060 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 433230 T2060 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 433230 T2035 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1389212302329/solrj_collection_shard2_replica1/data/index
   [junit4]   2> 433230 T2060 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 433231 T2035 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=38, maxMergeAtOnceExplicit=42, maxMergedSegmentMB=96.64453125, floorSegmentMB=2.1533203125, forceMergeDeletesPctAllowed=3.9670214619663904, segmentsPerTier=37.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 433231 T2060 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 433231 T2017 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 433231 T2060 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 433232 T2035 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/index3619636209tmp lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e9284e0),segFN=segments_1,generation=1}
   [junit4]   2> 433232 T2035 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 433232 T2060 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 433233 T2060 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 433233 T2060 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 433234 T2060 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 433234 T2017 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 433235 T2060 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 433235 T2060 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 433235 T2060 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 433235 T2017 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 433236 T2017 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 433236 T2060 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 433236 T2035 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 433237 T2060 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 433237 T2035 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 433236 T2017 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 433237 T2060 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 433237 T2035 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 433238 T2060 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 433238 T2049 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 433238 T2017 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 433238 T2035 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 433239 T2017 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 433239 T2035 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 433239 T2017 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 433239 T2035 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 433240 T2017 oasc.SolrCore.<init> [solrj_collection_shard1_replica2] Opening new SolrCore at ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1389212302085/solrj_collection_shard1_replica2/, dataDir=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1389212302085/solrj_collection_shard1_replica2/data/
   [junit4]   2> 433240 T2035 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 433240 T2017 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@46bcb6d2
   [junit4]   2> 433240 T2035 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 433241 T2035 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 433241 T2017 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1389212302085/solrj_collection_shard1_replica2/data
   [junit4]   2> 433241 T2035 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 433241 T2060 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 433241 T2017 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1389212302085/solrj_collection_shard1_replica2/data/index/
   [junit4]   2> 433241 T2035 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 433242 T2035 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 433242 T2017 oasc.SolrCore.initIndex WARN [solrj_collection_shard1_replica2] Solr index directory './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1389212302085/solrj_collection_shard1_replica2/data/index' doesn't exist. Creating new index...
   [junit4]   2> 433242 T2035 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 433243 T2035 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 433243 T2017 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1389212302085/solrj_collection_shard1_replica2/data/index
   [junit4]   2> 433243 T2035 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 433243 T2017 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=38, maxMergeAtOnceExplicit=42, maxMergedSegmentMB=96.64453125, floorSegmentMB=2.1533203125, forceMergeDeletesPctAllowed=3.9670214619663904, segmentsPerTier=37.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 433244 T2035 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 433244 T2035 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 433245 T2017 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/index3619636210tmp lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@14124673),segFN=segments_1,generation=1}
   [junit4]   2> 433245 T2017 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 433245 T2060 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 433245 T2035 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 433245 T2049 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 433245 T2060 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 433246 T2060 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=11, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=83.4619140625, floorSegmentMB=1.16015625, forceMergeDeletesPctAllowed=23.04746245212823, segmentsPerTier=34.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 433248 T2035 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 433248 T2035 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 433248 T2060 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/index3619636208tmp lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@64c837c5),segFN=segments_1,generation=1}
   [junit4]   2> 433248 T2035 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=11, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=83.4619140625, floorSegmentMB=1.16015625, forceMergeDeletesPctAllowed=23.04746245212823, segmentsPerTier=34.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 433249 T2049 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 433248 T2017 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 433248 T2060 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 433249 T2017 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 433250 T2035 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/index3619636209tmp lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e9284e0),segFN=segments_1,generation=1}
   [junit4]   2> 433250 T2060 oass.SolrIndexSearcher.<init> Opening Searcher@620cd265 main
   [junit4]   2> 433250 T2035 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 433250 T2017 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 433250 T2035 oass.SolrIndexSearcher.<init> Opening Searcher@487e6bb4 main
   [junit4]   2> 433251 T2017 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 433251 T2017 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 433252 T2017 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 433252 T2017 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 433252 T2017 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 433252 T2017 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 433253 T2087 oasc.SolrCore.registerSearcher [solrj_collection_shard1_replica1] Registered new searcher Searcher@620cd265 main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 433253 T2017 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 433253 T2088 oasc.SolrCore.registerSearcher [solrj_collection_shard2_replica1] Registered new searcher Searcher@487e6bb4 main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 433253 T2017 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 433254 T2017 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 433254 T2017 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 433254 T2017 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 433255 T2035 oasc.CoreContainer.registerCore registering core: solrj_collection_shard2_replica1
   [junit4]   2> 433255 T2017 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 433255 T2035 oasc.ZkController.register Register replica - core:solrj_collection_shard2_replica1 address:http://127.0.0.1:42305/io collection:solrj_collection shard:shard2
   [junit4]   2> 433255 T2017 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 433256 T2060 oasc.CoreContainer.registerCore registering core: solrj_collection_shard1_replica1
   [junit4]   2> 433256 T2060 oasc.ZkController.register Register replica - core:solrj_collection_shard1_replica1 address:http://127.0.0.1:40932/io collection:solrj_collection shard:shard1
   [junit4]   2> 433256 T2017 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 433256 T2035 oascc.SolrZkClient.makePath makePath: /collections/solrj_collection/leader_elect/shard2/election
   [junit4]   2> 433257 T2060 oascc.SolrZkClient.makePath makePath: /collections/solrj_collection/leader_elect/shard1/election
   [junit4]   2> 433258 T2017 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 433261 T2017 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 433261 T2017 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 433262 T2017 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=11, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=83.4619140625, floorSegmentMB=1.16015625, forceMergeDeletesPctAllowed=23.04746245212823, segmentsPerTier=34.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 433263 T2035 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
   [junit4]   2> 433263 T2060 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 433263 T2017 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/index3619636210tmp lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@14124673),segFN=segments_1,generation=1}
   [junit4]   2> 433264 T2017 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 433264 T2049 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 433264 T2017 oass.SolrIndexSearcher.<init> Opening Searcher@2aae2e24 main
   [junit4]   2> 433266 T2021 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 433266 T2021 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 433266 T2021 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 433266 T2091 oasc.SolrCore.registerSearcher [solrj_collection_shard1_replica2] Registered new searcher Searcher@2aae2e24 main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 433268 T2035 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up for shard shard2: total=2 found=1 timeoutin=-2
   [junit4]   2> 433268 T2035 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Was waiting for replicas to come up, but they are taking too long - assuming they won't come back till later
   [junit4]   2> 433269 T2035 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> 433269 T2060 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=-1
   [junit4]   2> 433269 T2060 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Was waiting for replicas to come up, but they are taking too long - assuming they won't come back till later
   [junit4]   2> 433270 T2060 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C607 name=solrj_collection_shard2_replica1 org.apache.solr.core.SolrCore@7d296a28 url=http://127.0.0.1:42305/io/solrj_collection_shard2_replica1 node=127.0.0.1:42305_io C607_STATE=coll:solrj_collection core:solrj_collection_shard2_replica1 props:{state=down, base_url=http://127.0.0.1:42305/io, core=solrj_collection_shard2_replica1, node_name=127.0.0.1:42305_io}
   [junit4]   2> 433269 T2035 C607 P42305 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:42305/io/solrj_collection_shard2_replica1/
   [junit4]   2> 433271 T2022 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> ASYNC  NEW_CORE C608 name=solrj_collection_shard1_replica1 org.apache.solr.core.SolrCore@648e570f url=http://127.0.0.1:40932/io/solrj_collection_shard1_replica1 node=127.0.0.1:40932_io C608_STATE=coll:solrj_collection core:solrj_collection_shard1_replica1 props:{state=down, base_url=http://127.0.0.1:40932/io, core=solrj_collection_shard1_replica1, node_name=127.0.0.1:40932_io}
   [junit4]   2> 433270 T2060 C608 P40932 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:40932/io/solrj_collection_shard1_replica1/
   [junit4]   2> 433270 T2017 oasc.CoreContainer.registerCore registering core: solrj_collection_shard1_replica2
   [junit4]   2> 433272 T2017 oasc.ZkController.register Register replica - core:solrj_collection_shard1_replica2 address:http://127.0.0.1:36776/io collection:solrj_collection shard:shard1
   [junit4]   2> 433271 T2049 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 433271 T2060 C608 P40932 oasu.PeerSync.sync PeerSync: core=solrj_collection_shard1_replica1 url=http://127.0.0.1:40932/io START replicas=[http://127.0.0.1:36776/io/solrj_collection_shard1_replica2/] nUpdates=100
   [junit4]   2> 433271 T2035 C607 P42305 oasu.PeerSync.sync PeerSync: core=solrj_collection_shard2_replica1 url=http://127.0.0.1:42305/io START replicas=[http://127.0.0.1:40584/io/solrj_collection_shard2_replica2/] nUpdates=100
   [junit4]   2> 433273 T2060 C608 P40932 oasu.PeerSync.sync PeerSync: core=solrj_collection_shard1_replica1 url=http://127.0.0.1:40932/io DONE.  We have no versions.  sync failed.
   [junit4]   2> 433274 T2060 C608 P40932 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 433274 T2035 C607 P42305 oasu.PeerSync.sync PeerSync: core=solrj_collection_shard2_replica1 url=http://127.0.0.1:42305/io DONE.  We have no versions.  sync failed.
   [junit4]   2> 433276 T2035 C607 P42305 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 433275 T2060 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 433277 T2060 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:40932/io/solrj_collection_shard1_replica1/ shard1
   [junit4]   2> 433277 T2035 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 433277 T2060 oascc.SolrZkClient.makePath makePath: /collections/solrj_collection/leaders/shard1
   [junit4]   2> 433277 T2035 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:42305/io/solrj_collection_shard2_replica1/ shard2
   [junit4]   2> 433278 T2021 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 433278 T2035 oascc.SolrZkClient.makePath makePath: /collections/solrj_collection/leaders/shard2
   [junit4]   2> 433280 T2049 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 433282 T2049 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 433282 T2049 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 433283 T2049 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> ASYNC  NEW_CORE C609 name=solrj_collection_shard1_replica2 org.apache.solr.core.SolrCore@596e84b2 url=http://127.0.0.1:36776/io/solrj_collection_shard1_replica2 node=127.0.0.1:36776_io C609_STATE=coll:solrj_collection core:solrj_collection_shard1_replica2 props:{state=down, base_url=http://127.0.0.1:36776/io, core=solrj_collection_shard1_replica2, node_name=127.0.0.1:36776_io}
   [junit4]   2> 433283 T2014 C609 P36776 oasc.SolrCore.execute [solrj_collection_shard1_replica2] webapp=/io path=/get params={getVersions=100&qt=/get&version=2&distrib=false&wt=javabin} status=0 QTime=1 
   [junit4]   2> 433284 T2021 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 433284 T2049 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 433284 T2049 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 433285 T2049 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 433285 T2021 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 433285 T2049 oasc.SolrCore.<init> [solrj_collection_shard2_replica2] Opening new SolrCore at ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1389212303500/solrj_collection_shard2_replica2/, dataDir=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1389212303500/solrj_col

[...truncated too long message...]

og ERROR There was a problem trying to register as the leader:org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections
   [junit4]   2> 		at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 		at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 		at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:205)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:202)
   [junit4]   2> 		at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:202)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:413)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:382)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:369)
   [junit4]   2> 		at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:112)
   [junit4]   2> 		at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:273)
   [junit4]   2> 		at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
   [junit4]   2> 		at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:107)
   [junit4]   2> 		at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 		at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:136)
   [junit4]   2> 		at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 		at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 	
   [junit4]   2> 563174 T2067 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/awholynewcollection_1/leader_elect/shard4/election/91043417413189644-core_node8-n_0000000002
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:173)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:170)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:170)
   [junit4]   2> 	at org.apache.solr.cloud.ElectionContext.cancelElection(ElectionContext.java:67)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:276)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:107)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:136)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 563174 T2067 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 563174 T2067 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 563175 T2067 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 563175 T2067 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 563175 T2067 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 563175 T2067 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 563175 T2067 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 563175 T2067 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 563175 T2067 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 563175 T2067 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 563175 T2067 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 563176 T2067 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 563176 T2067 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> NOTE: test params are: codec=Lucene46: {intDefault=PostingsFormat(name=NestedPulsing), text=PostingsFormat(name=Lucene41VarGapFixedInterval), range_facet_l=PostingsFormat(name=NestedPulsing), a_t=PostingsFormat(name=NestedPulsing), range_facet_sl=PostingsFormat(name=Lucene41VarGapFixedInterval), range_facet_si=PostingsFormat(name=Lucene41VarGapFixedInterval), multiDefault=PostingsFormat(name=Lucene41VarGapFixedInterval), a_si=PostingsFormat(name=Lucene41VarGapFixedInterval), _version_=PostingsFormat(name=NestedPulsing), id=PostingsFormat(name=Lucene41WithOrds), other_tl1=PostingsFormat(name=NestedPulsing), timestamp=PostingsFormat(name=NestedPulsing)}, docValues:{timestamp=DocValuesFormat(name=SimpleText)}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=ja_JP_JP_#u-ca-japanese, timezone=Asia/Tel_Aviv
   [junit4]   2> NOTE: Linux 3.8.0-35-generic amd64/Oracle Corporation 1.7.0_45 (64-bit)/cpus=8,threads=2,free=87848128,total=536870912
   [junit4]   2> NOTE: All tests run in this JVM: [AbstractAnalyticsStatsTest, SortByFunctionTest, DisMaxRequestHandlerTest, TestZkChroot, DateFieldTest, NoFacetTest, TestQuerySenderListener, MigrateRouteKeyTest, QueryFacetTest, SignatureUpdateProcessorFactoryTest, CurrencyFieldOpenExchangeTest, TestIndexSearcher, ChaosMonkeyNothingIsSafeTest, TestRTGBase, TestXIncludeConfig, TestDistributedSearch, TestSimpleQParserPlugin, ShardRoutingCustomTest, MultiTermTest, TestRecovery, CurrencyFieldXmlFileTest, HighlighterTest, AssignTest, TestBinaryField, TestStressRecovery, XsltUpdateRequestHandlerTest, DeleteReplicaTest, PluginInfoTest, TestFastLRUCache, ZkSolrClientTest, DistributedQueryComponentCustomSortTest, BlockDirectoryTest, TermVectorComponentDistributedTest, JSONWriterTest, TestMultiCoreConfBootstrap, TestCollationField, TestFieldTypeCollectionResource, TestManagedSchemaFieldResource, OpenExchangeRatesOrgProviderTest, RAMDirectoryFactoryTest, BadCopyFieldTest, TestStressLucene, PreAnalyzedUpdateProcessorTest, SuggesterTSTTest, TimeZoneUtilsTest, TestCopyFieldCollectionResource, RequiredFieldsTest, BadIndexSchemaTest, TestQuerySenderNoQuery, ExpressionTest, TestDistributedGrouping, PingRequestHandlerTest, DocumentAnalysisRequestHandlerTest, TestSolrQueryParserDefaultOperatorResource, LeaderElectionIntegrationTest, SolrCoreCheckLockOnStartupTest, TestRandomMergePolicy, TestSolr4Spatial, ZkNodePropsTest, SolrPluginUtilsTest, IndexBasedSpellCheckerTest, TestRangeQuery, UniqFieldsUpdateProcessorFactoryTest, TestJmxIntegration, TestFieldTypeResource, HardAutoCommitTest, TestIBSimilarityFactory, TestFieldCollectionResource, CachingDirectoryFactoryTest, RangeFacetTest, CollectionsAPIDistributedZkTest]
   [junit4] Completed on J1 in 136.92s, 1 test, 1 failure <<< FAILURES!

[...truncated 701 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:453: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:433: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:491: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1304: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:937: There were test failures: 362 suites, 1586 tests, 1 failure, 34 ignored (6 assumptions)

Total time: 46 minutes 22 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0_45 -XX:+UseCompressedOops -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure