You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2013/06/18 04:09:16 UTC
[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4073 - Still
Failing
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4073/
5 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
Error Message:
ERROR: SolrIndexSearcher opens=43 closes=42
Stack Trace:
java.lang.AssertionError: ERROR: SolrIndexSearcher opens=43 closes=42
at __randomizedtesting.SeedInfo.seed([620F21150099AD3]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:254)
at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:102)
at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:700)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
REGRESSION: org.apache.solr.cloud.ChaosMonkeyShardSplitTest.testDistribSearch
Error Message:
Server at http://127.0.0.1:14549 returned non ok status:500, message:Server Error
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:14549 returned non ok status:500, message:Server Error
at __randomizedtesting.SeedInfo.seed([620F21150099AD3:87C67C092756FAEF]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
at org.apache.solr.cloud.ChaosMonkeyShardSplitTest.doTest(ChaosMonkeyShardSplitTest.java:136)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
REGRESSION: org.apache.solr.cloud.RecoveryZkTest.testDistribSearch
Error Message:
expected:<208> but was:<38>
Stack Trace:
java.lang.AssertionError: expected:<208> but was:<38>
at __randomizedtesting.SeedInfo.seed([620F21150099AD3:87C67C092756FAEF]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.RecoveryZkTest.doTest(RecoveryZkTest.java:108)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
FAILED: org.apache.solr.cloud.ShardSplitTest.testDistribSearch
Error Message:
Server at http://127.0.0.1:60789/huwoy/w returned non ok status:500, message:Server Error
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:60789/huwoy/w returned non ok status:500, message:Server Error
at __randomizedtesting.SeedInfo.seed([620F21150099AD3:87C67C092756FAEF]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
at org.apache.solr.cloud.ShardSplitTest.doTest(ShardSplitTest.java:150)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
FAILED: org.apache.solr.cloud.SyncSliceTest.testDistribSearch
Error Message:
shard1 is not consistent. Got 305 from http://127.0.0.1:60805/_y/b/collection1lastClient and got 252 from http://127.0.0.1:60820/_y/b/collection1
Stack Trace:
java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:60805/_y/b/collection1lastClient and got 252 from http://127.0.0.1:60820/_y/b/collection1
at __randomizedtesting.SeedInfo.seed([620F21150099AD3:87C67C092756FAEF]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9485 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
[junit4:junit4] 2> 440949 T1519 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /icvy/dm
[junit4:junit4] 2> 440957 T1519 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./solrtest-ChaosMonkeySafeLeaderTest-1371519283114
[junit4:junit4] 2> 440958 T1519 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 440959 T1520 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 441060 T1519 oasc.ZkTestServer.run start zk server on port:21433
[junit4:junit4] 2> 441060 T1519 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 441069 T1526 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@473431f2 name:ZooKeeperConnection Watcher:127.0.0.1:21433 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 441069 T1519 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 441070 T1519 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 441082 T1519 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 441084 T1528 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@528ab79 name:ZooKeeperConnection Watcher:127.0.0.1:21433/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 441084 T1519 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 441085 T1519 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 441095 T1519 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 441098 T1519 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 441101 T1519 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 441104 T1519 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 441105 T1519 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 441119 T1519 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 441120 T1519 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 441223 T1519 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 441224 T1519 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 441227 T1519 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 441228 T1519 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 441231 T1519 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 441232 T1519 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 441235 T1519 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 441236 T1519 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 441244 T1519 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 441245 T1519 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 441248 T1519 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 441248 T1519 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 441251 T1519 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 441252 T1519 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 441525 T1519 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 441534 T1519 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:21436
[junit4:junit4] 2> 441535 T1519 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 441536 T1519 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 441537 T1519 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371519283418
[junit4:junit4] 2> 441537 T1519 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371519283418/solr.xml
[junit4:junit4] 2> 441538 T1519 oasc.CoreContainer.<init> New CoreContainer 804406828
[junit4:junit4] 2> 441539 T1519 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371519283418/'
[junit4:junit4] 2> 441539 T1519 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371519283418/'
[junit4:junit4] 2> 441668 T1519 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 441669 T1519 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 441670 T1519 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 441671 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 441671 T1519 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 441672 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 441672 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 441673 T1519 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 441673 T1519 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 441674 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 441692 T1519 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 441693 T1519 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:21433/solr
[junit4:junit4] 2> 441694 T1519 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 441695 T1519 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 441698 T1539 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3091b199 name:ZooKeeperConnection Watcher:127.0.0.1:21433 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 441699 T1519 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 441774 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 441787 T1519 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 441832 T1541 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@21cf41b5 name:ZooKeeperConnection Watcher:127.0.0.1:21433/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 441833 T1519 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 441897 T1519 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 442001 T1519 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 442035 T1519 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 442087 T1519 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:21436_icvy%2Fdm
[junit4:junit4] 2> 442089 T1519 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:21436_icvy%2Fdm
[junit4:junit4] 2> 442122 T1519 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 442144 T1519 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 442148 T1519 oasc.Overseer.start Overseer (id=89883887738486787-127.0.0.1:21436_icvy%2Fdm-n_0000000000) starting
[junit4:junit4] 2> 442160 T1519 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 442164 T1543 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 442165 T1519 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 442173 T1519 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 442175 T1519 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 442177 T1542 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 442181 T1544 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 442182 T1544 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 443680 T1542 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 443681 T1542 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:21436_icvy%2Fdm",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:21436/icvy/dm"}
[junit4:junit4] 2> 443681 T1542 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 443682 T1542 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 443686 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 444185 T1544 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371519283418/collection1
[junit4:junit4] 2> 444185 T1544 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 444186 T1544 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 444186 T1544 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 444188 T1544 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371519283418/collection1/'
[junit4:junit4] 2> 444189 T1544 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371519283418/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 444190 T1544 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371519283418/collection1/lib/README' to classloader
[junit4:junit4] 2> 444244 T1544 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 444306 T1544 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 444408 T1544 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 444415 T1544 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 445058 T1544 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 445072 T1544 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 445076 T1544 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 445090 T1544 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 445094 T1544 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 445099 T1544 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 445100 T1544 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 445100 T1544 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 445101 T1544 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 445102 T1544 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 445102 T1544 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 445102 T1544 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 445146 T1544 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371519283418/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/control/data/
[junit4:junit4] 2> 445146 T1544 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2b697029
[junit4:junit4] 2> 445147 T1544 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 445148 T1544 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/control/data
[junit4:junit4] 2> 445149 T1544 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/control/data/index/
[junit4:junit4] 2> 445150 T1544 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 445151 T1544 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/control/data/index
[junit4:junit4] 2> 445157 T1544 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/control/data/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 445158 T1544 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 445160 T1544 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 445161 T1544 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 445162 T1544 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 445162 T1544 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 445163 T1544 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 445163 T1544 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 445163 T1544 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 445164 T1544 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 445164 T1544 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 445179 T1544 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 445188 T1544 oass.SolrIndexSearcher.<init> Opening Searcher@61cc0eae main
[junit4:junit4] 2> 445190 T1544 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 445190 T1544 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 445193 T1545 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@61cc0eae main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 445195 T1544 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 445196 T1544 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:21436/icvy/dm collection:control_collection shard:shard1
[junit4:junit4] 2> 445196 T1544 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 445212 T1544 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 445223 T1544 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 445223 T1544 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 445223 T1544 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:21436/icvy/dm/collection1/
[junit4:junit4] 2> 445224 T1544 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 445224 T1544 oasc.SyncStrategy.syncToMe http://127.0.0.1:21436/icvy/dm/collection1/ has no replicas
[junit4:junit4] 2> 445224 T1544 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:21436/icvy/dm/collection1/
[junit4:junit4] 2> 445225 T1544 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 446700 T1542 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 446881 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 446921 T1544 oasc.ZkController.register We are http://127.0.0.1:21436/icvy/dm/collection1/ and leader is http://127.0.0.1:21436/icvy/dm/collection1/
[junit4:junit4] 2> 446921 T1544 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:21436/icvy/dm
[junit4:junit4] 2> 446921 T1544 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 446922 T1544 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 446922 T1544 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 446924 T1544 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 446925 T1519 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 446926 T1519 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 446927 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 446945 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 446947 T1519 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 446949 T1548 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@71a5bbf name:ZooKeeperConnection Watcher:127.0.0.1:21433/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 446949 T1519 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 446952 T1519 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 446954 T1519 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 447218 T1519 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 447221 T1519 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:21443
[junit4:junit4] 2> 447222 T1519 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 447223 T1519 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 447223 T1519 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371519289111
[junit4:junit4] 2> 447224 T1519 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371519289111/solr.xml
[junit4:junit4] 2> 447224 T1519 oasc.CoreContainer.<init> New CoreContainer 681309712
[junit4:junit4] 2> 447225 T1519 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371519289111/'
[junit4:junit4] 2> 447226 T1519 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371519289111/'
[junit4:junit4] 2> 447358 T1519 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 447359 T1519 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 447359 T1519 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 447360 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 447360 T1519 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 447361 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 447361 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 447362 T1519 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 447362 T1519 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 447363 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 447385 T1519 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 447385 T1519 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:21433/solr
[junit4:junit4] 2> 447386 T1519 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 447387 T1519 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 447390 T1559 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@20ef466c name:ZooKeeperConnection Watcher:127.0.0.1:21433 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 447390 T1519 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 447393 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 447406 T1519 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 447409 T1561 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@446bc840 name:ZooKeeperConnection Watcher:127.0.0.1:21433/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 447409 T1519 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 447414 T1519 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 448400 T1542 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 448401 T1542 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:21436_icvy%2Fdm",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:21436/icvy/dm"}
[junit4:junit4] 2> 448414 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 448414 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 448414 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 448417 T1519 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:21443_icvy%2Fdm
[junit4:junit4] 2> 448419 T1519 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:21443_icvy%2Fdm
[junit4:junit4] 2> 448427 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 448428 T1541 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 448428 T1561 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 448428 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 448428 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 448429 T1548 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 448435 T1562 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 448435 T1562 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 449919 T1542 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 449920 T1542 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:21443_icvy%2Fdm",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:21443/icvy/dm"}
[junit4:junit4] 2> 449921 T1542 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 3
[junit4:junit4] 2> 449921 T1542 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 449945 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 449945 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 449945 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 450439 T1562 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371519289111/collection1
[junit4:junit4] 2> 450439 T1562 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 450440 T1562 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 450440 T1562 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 450442 T1562 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371519289111/collection1/'
[junit4:junit4] 2> 450443 T1562 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371519289111/collection1/lib/README' to classloader
[junit4:junit4] 2> 450444 T1562 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371519289111/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 450498 T1562 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 450560 T1562 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 450661 T1562 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 450668 T1562 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 451328 T1562 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 451343 T1562 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 451347 T1562 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 451361 T1562 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 451366 T1562 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 451371 T1562 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 451372 T1562 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 451373 T1562 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 451374 T1562 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 451375 T1562 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 451375 T1562 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 451376 T1562 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 451376 T1562 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371519289111/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty1/
[junit4:junit4] 2> 451376 T1562 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2b697029
[junit4:junit4] 2> 451377 T1562 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 451378 T1562 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty1
[junit4:junit4] 2> 451379 T1562 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty1/index/
[junit4:junit4] 2> 451380 T1562 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 451381 T1562 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty1/index
[junit4:junit4] 2> 451387 T1562 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 451387 T1562 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 451391 T1562 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 451391 T1562 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 451392 T1562 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 451392 T1562 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 451393 T1562 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 451393 T1562 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 451394 T1562 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 451395 T1562 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 451395 T1562 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 451412 T1562 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 451422 T1562 oass.SolrIndexSearcher.<init> Opening Searcher@4521781b main
[junit4:junit4] 2> 451424 T1562 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 451424 T1562 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 451429 T1563 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4521781b main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 451431 T1562 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 451432 T1562 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:21443/icvy/dm collection:collection1 shard:shard1
[junit4:junit4] 2> 451433 T1562 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 451449 T1562 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 451458 T1562 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 451458 T1562 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 451459 T1562 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:21443/icvy/dm/collection1/
[junit4:junit4] 2> 451459 T1562 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 451459 T1562 oasc.SyncStrategy.syncToMe http://127.0.0.1:21443/icvy/dm/collection1/ has no replicas
[junit4:junit4] 2> 451459 T1562 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:21443/icvy/dm/collection1/
[junit4:junit4] 2> 451460 T1562 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 452959 T1542 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 453009 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 453009 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 453009 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 453061 T1562 oasc.ZkController.register We are http://127.0.0.1:21443/icvy/dm/collection1/ and leader is http://127.0.0.1:21443/icvy/dm/collection1/
[junit4:junit4] 2> 453061 T1562 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:21443/icvy/dm
[junit4:junit4] 2> 453062 T1562 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 453062 T1562 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 453062 T1562 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 453065 T1562 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 453066 T1519 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 453067 T1519 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 453067 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 453258 T1519 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 453260 T1519 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:21455
[junit4:junit4] 2> 453261 T1519 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 453262 T1519 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 453262 T1519 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371519295236
[junit4:junit4] 2> 453262 T1519 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371519295236/solr.xml
[junit4:junit4] 2> 453263 T1519 oasc.CoreContainer.<init> New CoreContainer 2076873335
[junit4:junit4] 2> 453263 T1519 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371519295236/'
[junit4:junit4] 2> 453264 T1519 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371519295236/'
[junit4:junit4] 2> 453353 T1519 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 453353 T1519 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 453354 T1519 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 453354 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 453354 T1519 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 453355 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 453355 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 453356 T1519 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 453356 T1519 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 453356 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 453368 T1519 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 453369 T1519 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:21433/solr
[junit4:junit4] 2> 453369 T1519 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 453370 T1519 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 453373 T1575 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4e72428a name:ZooKeeperConnection Watcher:127.0.0.1:21433 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 453374 T1519 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 453376 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 453385 T1519 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 453388 T1577 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5e0badf9 name:ZooKeeperConnection Watcher:127.0.0.1:21433/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 453388 T1519 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 453400 T1519 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 454403 T1519 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:21455_icvy%2Fdm
[junit4:junit4] 2> 454406 T1519 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:21455_icvy%2Fdm
[junit4:junit4] 2> 454409 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 454409 T1577 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 454409 T1541 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 454409 T1561 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 454410 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 454410 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 454411 T1548 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 454417 T1578 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 454417 T1578 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 454523 T1542 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 454524 T1542 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:21443_icvy%2Fdm",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:21443/icvy/dm"}
[junit4:junit4] 2> 454529 T1542 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:21455_icvy%2Fdm",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:21455/icvy/dm"}
[junit4:junit4] 2> 454529 T1542 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4] 2> 454530 T1542 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 454547 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 454547 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 454547 T1577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 454547 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 455420 T1578 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371519295236/collection1
[junit4:junit4] 2> 455420 T1578 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 455421 T1578 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 455421 T1578 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 455423 T1578 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371519295236/collection1/'
[junit4:junit4] 2> 455425 T1578 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371519295236/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 455425 T1578 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371519295236/collection1/lib/README' to classloader
[junit4:junit4] 2> 455480 T1578 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 455542 T1578 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 455644 T1578 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 455653 T1578 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 456305 T1578 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 456320 T1578 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 456324 T1578 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 456339 T1578 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 456344 T1578 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 456349 T1578 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 456351 T1578 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 456352 T1578 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 456352 T1578 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 456353 T1578 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 456353 T1578 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 456354 T1578 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 456354 T1578 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371519295236/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty2/
[junit4:junit4] 2> 456355 T1578 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2b697029
[junit4:junit4] 2> 456355 T1578 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 456356 T1578 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty2
[junit4:junit4] 2> 456357 T1578 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty2/index/
[junit4:junit4] 2> 456358 T1578 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 456359 T1578 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty2/index
[junit4:junit4] 2> 456365 T1578 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty2/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 456366 T1578 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 456369 T1578 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 456369 T1578 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 456370 T1578 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 456371 T1578 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 456372 T1578 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 456372 T1578 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 456372 T1578 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 456373 T1578 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 456373 T1578 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 456390 T1578 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 456401 T1578 oass.SolrIndexSearcher.<init> Opening Searcher@221391ab main
[junit4:junit4] 2> 456402 T1578 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 456402 T1578 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 456407 T1579 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@221391ab main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 456409 T1578 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 456409 T1578 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:21455/icvy/dm collection:collection1 shard:shard2
[junit4:junit4] 2> 456410 T1578 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 456426 T1578 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 456434 T1578 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 456435 T1578 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 456435 T1578 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:21455/icvy/dm/collection1/
[junit4:junit4] 2> 456435 T1578 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 456436 T1578 oasc.SyncStrategy.syncToMe http://127.0.0.1:21455/icvy/dm/collection1/ has no replicas
[junit4:junit4] 2> 456436 T1578 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:21455/icvy/dm/collection1/
[junit4:junit4] 2> 456436 T1578 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 457555 T1542 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 457601 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 457601 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 457601 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 457601 T1577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 457618 T1578 oasc.ZkController.register We are http://127.0.0.1:21455/icvy/dm/collection1/ and leader is http://127.0.0.1:21455/icvy/dm/collection1/
[junit4:junit4] 2> 457618 T1578 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:21455/icvy/dm
[junit4:junit4] 2> 457618 T1578 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 457619 T1578 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 457619 T1578 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 457621 T1578 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 457622 T1519 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 457623 T1519 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 457624 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 457905 T1519 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 457909 T1519 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:21458
[junit4:junit4] 2> 457911 T1519 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 457912 T1519 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 457912 T1519 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371519299798
[junit4:junit4] 2> 457913 T1519 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371519299798/solr.xml
[junit4:junit4] 2> 457913 T1519 oasc.CoreContainer.<init> New CoreContainer 973323550
[junit4:junit4] 2> 457914 T1519 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371519299798/'
[junit4:junit4] 2> 457915 T1519 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371519299798/'
[junit4:junit4] 2> 458063 T1519 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 458064 T1519 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 458065 T1519 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 458065 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 458066 T1519 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 458067 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 458067 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 458068 T1519 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 458068 T1519 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 458069 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 458087 T1519 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 458088 T1519 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:21433/solr
[junit4:junit4] 2> 458088 T1519 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 458089 T1519 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 458092 T1591 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@719a9897 name:ZooKeeperConnection Watcher:127.0.0.1:21433 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 458093 T1519 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 458104 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 458117 T1519 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 458119 T1593 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2ac0d168 name:ZooKeeperConnection Watcher:127.0.0.1:21433/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 458119 T1519 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 458128 T1519 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 459114 T1542 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 459115 T1542 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:21455_icvy%2Fdm",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:21455/icvy/dm"}
[junit4:junit4] 2> 459132 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 459132 T1519 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:21458_icvy%2Fdm
[junit4:junit4] 2> 459132 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 459132 T1593 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 459132 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 459132 T1577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 459140 T1519 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:21458_icvy%2Fdm
[junit4:junit4] 2> 459143 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 459143 T1593 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 459143 T1577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 459143 T1561 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 459143 T1541 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 459144 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 459144 T1548 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 459144 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 459145 T1593 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 459145 T1577 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 459151 T1594 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 459151 T1594 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 460642 T1542 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 460644 T1542 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:21458_icvy%2Fdm",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:21458/icvy/dm"}
[junit4:junit4] 2> 460644 T1542 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4] 2> 460644 T1542 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard3
[junit4:junit4] 2> 460649 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 460649 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 460649 T1593 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 460649 T1577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 460649 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 461155 T1594 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371519299798/collection1
[junit4:junit4] 2> 461155 T1594 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 461156 T1594 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 461156 T1594 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 461158 T1594 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371519299798/collection1/'
[junit4:junit4] 2> 461160 T1594 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371519299798/collection1/lib/README' to classloader
[junit4:junit4] 2> 461160 T1594 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371519299798/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 461213 T1594 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 461275 T1594 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 461377 T1594 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 461384 T1594 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 462041 T1594 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 462055 T1594 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 462060 T1594 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 462075 T1594 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 462080 T1594 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 462085 T1594 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 462086 T1594 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 462087 T1594 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 462087 T1594 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 462088 T1594 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 462089 T1594 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 462089 T1594 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 462090 T1594 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371519299798/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty3/
[junit4:junit4] 2> 462090 T1594 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2b697029
[junit4:junit4] 2> 462091 T1594 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 462092 T1594 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty3
[junit4:junit4] 2> 462093 T1594 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty3/index/
[junit4:junit4] 2> 462094 T1594 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 462094 T1594 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty3/index
[junit4:junit4] 2> 462101 T1594 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty3/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 462101 T1594 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 462105 T1594 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 462105 T1594 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 462106 T1594 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 462107 T1594 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 462107 T1594 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 462108 T1594 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 462108 T1594 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 462109 T1594 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 462109 T1594 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 462126 T1594 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 462136 T1594 oass.SolrIndexSearcher.<init> Opening Searcher@13bc5157 main
[junit4:junit4] 2> 462137 T1594 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 462137 T1594 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 462142 T1595 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@13bc5157 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 462145 T1594 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 462145 T1594 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:21458/icvy/dm collection:collection1 shard:shard3
[junit4:junit4] 2> 462146 T1594 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4:junit4] 2> 462168 T1594 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 462176 T1594 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 462177 T1594 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 462177 T1594 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:21458/icvy/dm/collection1/
[junit4:junit4] 2> 462177 T1594 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 462178 T1594 oasc.SyncStrategy.syncToMe http://127.0.0.1:21458/icvy/dm/collection1/ has no replicas
[junit4:junit4] 2> 462178 T1594 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:21458/icvy/dm/collection1/
[junit4:junit4] 2> 462178 T1594 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3
[junit4:junit4] 2> 463670 T1542 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 463716 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 463716 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 463716 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 463716 T1577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 463717 T1593 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 463772 T1594 oasc.ZkController.register We are http://127.0.0.1:21458/icvy/dm/collection1/ and leader is http://127.0.0.1:21458/icvy/dm/collection1/
[junit4:junit4] 2> 463772 T1594 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:21458/icvy/dm
[junit4:junit4] 2> 463773 T1594 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 463773 T1594 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 463773 T1594 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 463775 T1594 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 463776 T1519 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 463777 T1519 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 463778 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 464118 T1519 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 464122 T1519 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:21464
[junit4:junit4] 2> 464123 T1519 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 464124 T1519 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 464124 T1519 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371519305951
[junit4:junit4] 2> 464125 T1519 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371519305951/solr.xml
[junit4:junit4] 2> 464125 T1519 oasc.CoreContainer.<init> New CoreContainer 1942403182
[junit4:junit4] 2> 464126 T1519 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371519305951/'
[junit4:junit4] 2> 464127 T1519 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371519305951/'
[junit4:junit4] 2> 464300 T1519 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 464301 T1519 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 464302 T1519 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 464302 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 464303 T1519 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 464303 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 464304 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 464304 T1519 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 464305 T1519 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 464305 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 464325 T1519 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 464327 T1519 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:21433/solr
[junit4:junit4] 2> 464327 T1519 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 464328 T1519 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 464332 T1607 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@10e469e name:ZooKeeperConnection Watcher:127.0.0.1:21433 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 464332 T1519 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 464344 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 464357 T1519 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 464360 T1609 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1236ab85 name:ZooKeeperConnection Watcher:127.0.0.1:21433/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 464360 T1519 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 464366 T1519 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 465230 T1542 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 465231 T1542 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard3",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:21458_icvy%2Fdm",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:21458/icvy/dm"}
[junit4:junit4] 2> 465266 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 465267 T1593 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 465267 T1609 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 465266 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 465266 T1577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 465266 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 465370 T1519 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:21464_icvy%2Fdm
[junit4:junit4] 2> 465372 T1519 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:21464_icvy%2Fdm
[junit4:junit4] 2> 465374 T1577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 465374 T1593 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 465374 T1609 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 465374 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 465375 T1609 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 465374 T1561 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 465374 T1541 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 465376 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 465375 T1577 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 465376 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 465383 T1593 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 465384 T1548 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 465389 T1610 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 465389 T1610 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 466772 T1542 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 466773 T1542 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:21464_icvy%2Fdm",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:21464/icvy/dm"}
[junit4:junit4] 2> 466774 T1542 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4] 2> 466774 T1542 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 466978 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 466978 T1609 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 466978 T1593 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 466978 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 466978 T1577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 466978 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 467392 T1610 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371519305951/collection1
[junit4:junit4] 2> 467392 T1610 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 467393 T1610 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 467393 T1610 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 467395 T1610 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371519305951/collection1/'
[junit4:junit4] 2> 467396 T1610 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371519305951/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 467397 T1610 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371519305951/collection1/lib/README' to classloader
[junit4:junit4] 2> 467449 T1610 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 467510 T1610 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 467512 T1610 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 467519 T1610 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 468133 T1610 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 468143 T1610 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 468146 T1610 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 468158 T1610 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 468161 T1610 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 468165 T1610 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 468166 T1610 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 468167 T1610 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 468167 T1610 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 468168 T1610 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 468168 T1610 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 468168 T1610 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 468169 T1610 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371519305951/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty4/
[junit4:junit4] 2> 468169 T1610 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2b697029
[junit4:junit4] 2> 468169 T1610 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 468170 T1610 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty4
[junit4:junit4] 2> 468171 T1610 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty4/index/
[junit4:junit4] 2> 468171 T1610 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 468172 T1610 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty4/index
[junit4:junit4] 2> 468176 T1610 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty4/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 468177 T1610 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 468179 T1610 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 468179 T1610 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 468180 T1610 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 468180 T1610 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 468181 T1610 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 468181 T1610 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 468181 T1610 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 468182 T1610 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 468182 T1610 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 468192 T1610 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 468199 T1610 oass.SolrIndexSearcher.<init> Opening Searcher@715b66a4 main
[junit4:junit4] 2> 468200 T1610 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 468200 T1610 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 468204 T1611 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@715b66a4 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 468205 T1610 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 468206 T1610 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:21464/icvy/dm collection:collection1 shard:shard1
[junit4:junit4] 2> 468216 T1610 oasc.ZkController.register We are http://127.0.0.1:21464/icvy/dm/collection1/ and leader is http://127.0.0.1:21443/icvy/dm/collection1/
[junit4:junit4] 2> 468216 T1610 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:21464/icvy/dm
[junit4:junit4] 2> 468216 T1610 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 468217 T1610 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C502 name=collection1 org.apache.solr.core.SolrCore@473a2e60 url=http://127.0.0.1:21464/icvy/dm/collection1 node=127.0.0.1:21464_icvy%2Fdm C502_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:21464_icvy%2Fdm, base_url=http://127.0.0.1:21464/icvy/dm}
[junit4:junit4] 2> 468217 T1612 C502 P21464 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 468218 T1612 C502 P21464 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 468218 T1610 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 468218 T1612 C502 P21464 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 468218 T1612 C502 P21464 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 468219 T1519 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 468219 T1519 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 468220 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 468223 T1612 C502 P21464 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 468234 T1551 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 468405 T1519 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 468407 T1519 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:21471
[junit4:junit4] 2> 468408 T1519 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 468408 T1519 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 468409 T1519 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371519310389
[junit4:junit4] 2> 468409 T1519 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371519310389/solr.xml
[junit4:junit4] 2> 468409 T1519 oasc.CoreContainer.<init> New CoreContainer 1876473000
[junit4:junit4] 2> 468410 T1519 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371519310389/'
[junit4:junit4] 2> 468410 T1519 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371519310389/'
[junit4:junit4] 2> 468496 T1519 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 468496 T1519 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 468497 T1519 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 468497 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 468497 T1519 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 468498 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 468498 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 468499 T1519 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 468499 T1519 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 468499 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 468508 T1542 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 468509 T1542 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:21464_icvy%2Fdm",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:21464/icvy/dm"}
[junit4:junit4] 2> 468511 T1519 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 468512 T1519 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:21433/solr
[junit4:junit4] 2> 468512 T1519 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 468513 T1519 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 468522 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 468522 T1593 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 468522 T1609 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 468522 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 468522 T1577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 468522 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 468523 T1624 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4923a958 name:ZooKeeperConnection Watcher:127.0.0.1:21433 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 468524 T1519 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 468533 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 468542 T1519 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 468544 T1626 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@63bf2e9e name:ZooKeeperConnection Watcher:127.0.0.1:21433/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 468544 T1519 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 468549 T1519 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 469236 T1551 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 469236 T1551 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=5&state=recovering&nodeName=127.0.0.1:21464_icvy%252Fdm&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1002
[junit4:junit4] 2> 469552 T1519 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:21471_icvy%2Fdm
[junit4:junit4] 2> 469554 T1519 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:21471_icvy%2Fdm
[junit4:junit4] 2> 469556 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 469556 T1593 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 469556 T1577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 469558 T1626 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4] 2> 469558 T1561 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4] 2> 469558 T1609 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4] 2> 469558 T1541 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4] 2> 469559 T1609 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 469559 T1548 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4] 2> 469559 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 469560 T1593 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4] 2> 469560 T1577 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4] 2> 469559 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 469565 T1627 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 469565 T1627 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 470028 T1542 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 470029 T1542 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"6",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:21471_icvy%2Fdm",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:21471/icvy/dm"}
[junit4:junit4] 2> 470029 T1542 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4] 2> 470030 T1542 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 470034 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 470034 T1626 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 470034 T1593 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 470034 T1609 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 470034 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 470034 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 470034 T1577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 470568 T1627 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371519310389/collection1
[junit4:junit4] 2> 470568 T1627 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 470569 T1627 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 470569 T1627 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 470571 T1627 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371519310389/collection1/'
[junit4:junit4] 2> 470572 T1627 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371519310389/collection1/lib/README' to classloader
[junit4:junit4] 2> 470573 T1627 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371519310389/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 470628 T1627 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 470693 T1627 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 470695 T1627 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 470701 T1627 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> ASYNC NEW_CORE C503 name=collection1 org.apache.solr.core.SolrCore@473a2e60 url=http://127.0.0.1:21464/icvy/dm/collection1 node=127.0.0.1:21464_icvy%2Fdm C503_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:21464_icvy%2Fdm, base_url=http://127.0.0.1:21464/icvy/dm}
[junit4:junit4] 2> 471238 T1612 C503 P21464 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:21443/icvy/dm/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 471238 T1612 C503 P21464 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:21464/icvy/dm START replicas=[http://127.0.0.1:21443/icvy/dm/collection1/] nUpdates=100
[junit4:junit4] 2> 471239 T1612 C503 P21464 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 471241 T1612 C503 P21464 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 471241 T1612 C503 P21464 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 471241 T1612 C503 P21464 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 471241 T1612 C503 P21464 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 471242 T1612 C503 P21464 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:21443/icvy/dm/collection1/. core=collection1
[junit4:junit4] 2> 471242 T1612 C503 P21464 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C504 name=collection1 org.apache.solr.core.SolrCore@3ddeee1a url=http://127.0.0.1:21443/icvy/dm/collection1 node=127.0.0.1:21443_icvy%2Fdm C504_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:21443_icvy%2Fdm, base_url=http://127.0.0.1:21443/icvy/dm, leader=true}
[junit4:junit4] 2> 471250 T1552 C504 P21443 oasc.SolrCore.execute [collection1] webapp=/icvy/dm path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 471256 T1553 C504 P21443 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 471262 T1553 C504 P21443 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 471263 T1553 C504 P21443 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 471267 T1553 C504 P21443 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 471267 T1553 C504 P21443 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 471269 T1553 C504 P21443 oass.SolrIndexSearcher.<init> Opening Searcher@19b63536 realtime
[junit4:junit4] 2> 471270 T1553 C504 P21443 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 471270 T1553 C504 P21443 oasup.LogUpdateProcessor.finish [collection1] webapp=/icvy/dm path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 14
[junit4:junit4] 2> 471271 T1612 C503 P21464 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 471272 T1612 C503 P21464 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 471275 T1554 C504 P21443 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 471275 T1554 C504 P21443 oasc.SolrCore.execute [collection1] webapp=/icvy/dm path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2
[junit4:junit4] 2> 471276 T1612 C503 P21464 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 471276 T1612 C503 P21464 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 471277 T1612 C503 P21464 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 471280 T1554 C504 P21443 oasc.SolrCore.execute [collection1] webapp=/icvy/dm path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1
[junit4:junit4] 2> 471281 T1612 C503 P21464 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 471283 T1612 C503 P21464 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty4/index.20130618023513438
[junit4:junit4] 2> 471284 T1612 C503 P21464 oash.SnapPuller.fetchLatestIndex Starting download to org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty4/index.20130618023513438 lockFactory=org.apache.lucene.store.NativeFSLockFactory@196972e8 fullCopy=false
[junit4:junit4] 2> 471288 T1554 C504 P21443 oasc.SolrCore.execute [collection1] webapp=/icvy/dm path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 471290 T1612 C503 P21464 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 471294 T1612 C503 P21464 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 471294 T1612 C503 P21464 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 471300 T1612 C503 P21464 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty4/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty4/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 471300 T1612 C503 P21464 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 471301 T1612 C503 P21464 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 471302 T1612 C503 P21464 oass.SolrIndexSearcher.<init> Opening Searcher@765f73a7 main
[junit4:junit4] 2> 471303 T1611 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@765f73a7 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 471304 T1612 C503 P21464 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty4/index.20130618023513438 [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty4/index.20130618023513438;done=true>>]
[junit4:junit4] 2> 471304 T1612 C503 P21464 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty4/index.20130618023513438
[junit4:junit4] 2> 471305 T1612 C503 P21464 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty4/index.20130618023513438
[junit4:junit4] 2> 471306 T1612 C503 P21464 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 471306 T1612 C503 P21464 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 471307 T1612 C503 P21464 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 471307 T1612 C503 P21464 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 471309 T1612 C503 P21464 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 471364 T1627 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 471377 T1627 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 471382 T1627 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 471396 T1627 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 471415 T1627 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 471420 T1627 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 471421 T1627 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 471422 T1627 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 471422 T1627 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 471423 T1627 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 471424 T1627 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 471424 T1627 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 471425 T1627 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371519310389/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty5/
[junit4:junit4] 2> 471425 T1627 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2b697029
[junit4:junit4] 2> 471426 T1627 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 471427 T1627 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty5
[junit4:junit4] 2> 471428 T1627 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty5/index/
[junit4:junit4] 2> 471428 T1627 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty5/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 471429 T1627 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty5/index
[junit4:junit4] 2> 471435 T1627 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty5/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 471435 T1627 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 471439 T1627 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 471439 T1627 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 471440 T1627 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 471441 T1627 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 471441 T1627 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 471441 T1627 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 471445 T1627 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 471446 T1627 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 471447 T1627 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 471463 T1627 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 471472 T1627 oass.SolrIndexSearcher.<init> Opening Searcher@46675088 main
[junit4:junit4] 2> 471474 T1627 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 471475 T1627 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 471481 T1630 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@46675088 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 471484 T1627 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 471484 T1627 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:21471/icvy/dm collection:collection1 shard:shard2
[junit4:junit4] 2> 471547 T1627 oasc.ZkController.register We are http://127.0.0.1:21471/icvy/dm/collection1/ and leader is http://127.0.0.1:21455/icvy/dm/collection1/
[junit4:junit4] 2> 471547 T1542 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 471548 T1627 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:21471/icvy/dm
[junit4:junit4] 2> 471548 T1627 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 471548 T1627 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> 471548 T1542 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:21464_icvy%2Fdm",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:21464/icvy/dm"}
[junit4:junit4] 2> ASYNC NEW_CORE C505 name=collection1 org.apache.solr.core.SolrCore@162ccb7e url=http://127.0.0.1:21471/icvy/dm/collection1 node=127.0.0.1:21471_icvy%2Fdm C505_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:21471_icvy%2Fdm, base_url=http://127.0.0.1:21471/icvy/dm}
[junit4:junit4] 2> 471549 T1631 C505 P21471 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 471549 T1627 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 471550 T1631 C505 P21471 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 471550 T1631 C505 P21471 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 471551 T1631 C505 P21471 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 471592 T1519 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 471593 T1519 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 471593 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 471616 T1631 C505 P21471 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 471630 T1567 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 6, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 471693 T1542 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"6",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:21471_icvy%2Fdm",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:21471/icvy/dm"}
[junit4:junit4] 2> 471882 T1519 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 471886 T1519 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:21483
[junit4:junit4] 2> 471886 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 471886 T1593 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 471886 T1577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 471886 T1626 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 471886 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 471886 T1609 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 471886 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 471887 T1519 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 471889 T1519 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 471890 T1519 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371519313768
[junit4:junit4] 2> 471890 T1519 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371519313768/solr.xml
[junit4:junit4] 2> 471891 T1519 oasc.CoreContainer.<init> New CoreContainer 1007048722
[junit4:junit4] 2> 471892 T1519 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371519313768/'
[junit4:junit4] 2> 471892 T1519 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371519313768/'
[junit4:junit4] 2> 472022 T1519 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 472023 T1519 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 472024 T1519 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 472024 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 472025 T1519 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 472025 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 472026 T1519 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 472026 T1519 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 472027 T1519 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 472028 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 472046 T1519 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 472047 T1519 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:21433/solr
[junit4:junit4] 2> 472047 T1519 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 472049 T1519 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 472051 T1643 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@757ae69c name:ZooKeeperConnection Watcher:127.0.0.1:21433 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 472052 T1519 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 472055 T1519 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 472068 T1519 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 472070 T1645 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@39c8278b name:ZooKeeperConnection Watcher:127.0.0.1:21433/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 472070 T1519 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 472075 T1519 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 472632 T1567 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 6, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 472632 T1567 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=6&state=recovering&nodeName=127.0.0.1:21471_icvy%252Fdm&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1002
[junit4:junit4] 2> 473078 T1519 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:21483_icvy%2Fdm
[junit4:junit4] 2> 473090 T1519 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:21483_icvy%2Fdm
[junit4:junit4] 2> 473093 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 473094 T1626 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 473094 T1645 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4] 2> 473094 T1609 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4] 2> 473094 T1541 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4] 2> 473094 T1561 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4] 2> 473094 T1593 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 473095 T1548 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4] 2> 473094 T1577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 473095 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 473095 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 473095 T1609 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 473096 T1626 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4] 2> 473097 T1577 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4] 2> 473097 T1593 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4] 2> 473102 T1646 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 473102 T1646 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 473416 T1542 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 473417 T1542 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"7",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:21483_icvy%2Fdm",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:21483/icvy/dm"}
[junit4:junit4] 2> 473417 T1542 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4] 2> 473417 T1542 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard3
[junit4:junit4] 2> 473430 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 473430 T1593 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 473430 T1609 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 473430 T1645 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 473430 T1626 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 473430 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 473430 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 473430 T1577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 474104 T1646 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371519313768/collection1
[junit4:junit4] 2> 474104 T1646 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 474105 T1646 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 474105 T1646 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 474107 T1646 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371519313768/collection1/'
[junit4:junit4] 2> 474108 T1646 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371519313768/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 474109 T1646 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371519313768/collection1/lib/README' to classloader
[junit4:junit4] 2> 474161 T1646 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 474222 T1646 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 474323 T1646 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 474329 T1646 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> ASYNC NEW_CORE C506 name=collection1 org.apache.solr.core.SolrCore@162ccb7e url=http://127.0.0.1:21471/icvy/dm/collection1 node=127.0.0.1:21471_icvy%2Fdm C506_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:21471_icvy%2Fdm, base_url=http://127.0.0.1:21471/icvy/dm}
[junit4:junit4] 2> 474634 T1631 C506 P21471 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:21455/icvy/dm/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 474634 T1631 C506 P21471 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:21471/icvy/dm START replicas=[http://127.0.0.1:21455/icvy/dm/collection1/] nUpdates=100
[junit4:junit4] 2> 474635 T1631 C506 P21471 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 474635 T1631 C506 P21471 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 474635 T1631 C506 P21471 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 474635 T1631 C506 P21471 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 474636 T1631 C506 P21471 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 474636 T1631 C506 P21471 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:21455/icvy/dm/collection1/. core=collection1
[junit4:junit4] 2> 474637 T1631 C506 P21471 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C507 name=collection1 org.apache.solr.core.SolrCore@314e6ebd url=http://127.0.0.1:21455/icvy/dm/collection1 node=127.0.0.1:21455_icvy%2Fdm C507_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:21455_icvy%2Fdm, base_url=http://127.0.0.1:21455/icvy/dm, leader=true}
[junit4:junit4] 2> 474645 T1568 C507 P21455 oasc.SolrCore.execute [collection1] webapp=/icvy/dm path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 474650 T1569 C507 P21455 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 474656 T1569 C507 P21455 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty2/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 474656 T1569 C507 P21455 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 474660 T1569 C507 P21455 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty2/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty2/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 474660 T1569 C507 P21455 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 474662 T1569 C507 P21455 oass.SolrIndexSearcher.<init> Opening Searcher@73ceca9f realtime
[junit4:junit4] 2> 474663 T1569 C507 P21455 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 474663 T1569 C507 P21455 oasup.LogUpdateProcessor.finish [collection1] webapp=/icvy/dm path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 13
[junit4:junit4] 2> 474664 T1631 C506 P21471 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 474665 T1631 C506 P21471 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 474667 T1570 C507 P21455 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 474667 T1570 C507 P21455 oasc.SolrCore.execute [collection1] webapp=/icvy/dm path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 474668 T1631 C506 P21471 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 474668 T1631 C506 P21471 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 474669 T1631 C506 P21471 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 474672 T1570 C507 P21455 oasc.SolrCore.execute [collection1] webapp=/icvy/dm path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1
[junit4:junit4] 2> 474673 T1631 C506 P21471 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 474675 T1631 C506 P21471 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty5/index.20130618023516830
[junit4:junit4] 2> 474676 T1631 C506 P21471 oash.SnapPuller.fetchLatestIndex Starting download to org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty5/index.20130618023516830 lockFactory=org.apache.lucene.store.NativeFSLockFactory@53ea2e17 fullCopy=false
[junit4:junit4] 2> 474680 T1570 C507 P21455 oasc.SolrCore.execute [collection1] webapp=/icvy/dm path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 474682 T1631 C506 P21471 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 474685 T1631 C506 P21471 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 474685 T1631 C506 P21471 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 474691 T1631 C506 P21471 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty5/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty5/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 474692 T1631 C506 P21471 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 474692 T1631 C506 P21471 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 474694 T1631 C506 P21471 oass.SolrIndexSearcher.<init> Opening Searcher@315471c7 main
[junit4:junit4] 2> 474695 T1630 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@315471c7 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 474696 T1631 C506 P21471 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty5/index.20130618023516830 [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty5/index.20130618023516830;done=true>>]
[junit4:junit4] 2> 474696 T1631 C506 P21471 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty5/index.20130618023516830
[junit4:junit4] 2> 474696 T1631 C506 P21471 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty5/index.20130618023516830
[junit4:junit4] 2> 474698 T1631 C506 P21471 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 474698 T1631 C506 P21471 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 474698 T1631 C506 P21471 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 474698 T1631 C506 P21471 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 474700 T1631 C506 P21471 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 474934 T1542 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 474935 T1542 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"6",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:21471_icvy%2Fdm",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:21471/icvy/dm"}
[junit4:junit4] 2> 474950 T1548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 474950 T1626 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 474950 T1609 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 474950 T1593 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 474950 T1541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 474950 T1561 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 474950 T1577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 474950 T1645 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 474988 T1646 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 475001 T1646 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 475006 T1646 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 475019 T1646 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 475039 T1646 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 475043 T1646 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 475044 T1646 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 475045 T1646 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 475045 T1646 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 475046 T1646 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 475047 T1646 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 475047 T1646 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 475048 T1646 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371519313768/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty6/
[junit4:junit4] 2> 475048 T1646 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2b697029
[junit4:junit4] 2> 475048 T1646 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 475050 T1646 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty6
[junit4:junit4] 2> 475050 T1646 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty6/index/
[junit4:junit4] 2> 475051 T1646 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty6/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 475052 T1646 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty6/index
[junit4:junit4] 2> 475058 T1646 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371519283113/jetty6/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 475059 T1646 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 475062 T1646 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 475062 T1646 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 475063 T1646 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 475064 T1646 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 475064 T1646 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 475065 T1646 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 475068 T1646 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 475069 T1646 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 475069 T1646 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 475086 T1646 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 475094 T1646 oass.SolrIndexSearcher.<init> Opening Searcher@25368798 main
[junit4:junit4] 2> 475096 T1646 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 475096 T1646 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 475100 T1649 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@25368798 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 475103 T1646 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 475103 T1646 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:21483/icvy/dm collection:collection1 shard:shard3
[junit4:junit4] 2> 475106 T1646 oasc.ZkController.register We are http://127.0.0.1:21483/icvy/dm/collection1/ and leader is http://127.0.0.1:21458/icvy/dm/collection1/
[junit4:junit4] 2> 475106 T1646 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:21483/icvy/dm
[junit4:junit4] 2> 475106 T1646 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 475107 T1646 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C508 name=collection1 org.apache.solr.core.Solr
[...truncated too long message...]
achedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371521047093/jetty4/index;done=false>>]
[junit4:junit4] 2> 2500254 T3983 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371521047093/jetty4/index
[junit4:junit4] 2> 2500255 T3983 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371521047093/jetty4 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371521047093/jetty4;done=false>>]
[junit4:junit4] 2> 2500255 T3983 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371521047093/jetty4
[junit4:junit4] 2> 2500255 T3983 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 2500256 T3983 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard2/election/89884003342942220-5-n_0000000001
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4] 2> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:152)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:149)
[junit4:junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:149)
[junit4:junit4] 2> at org.apache.solr.cloud.ElectionContext.cancelElection(ElectionContext.java:63)
[junit4:junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:260)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:156)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:100)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4] 2>
[junit4:junit4] 2> 2500257 T3983 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 2500257 T3983 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2500257 T3983 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 2500257 T3983 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2500258 T3983 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 2500258 T3983 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:235)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:232)
[junit4:junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:232)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4] 2>
[junit4:junit4] 2> NOTE: test params are: codec=Lucene40, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=en_SG, timezone=Asia/Rangoon
[junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=3,free=193006808,total=536870912
[junit4:junit4] 2> NOTE: All tests run in this JVM: [EchoParamsTest, TestCopyFieldCollectionResource, TestSolrXMLSerializer, TestMaxScoreQueryParser, TestCSVResponseWriter, TestDynamicFieldCollectionResource, ClusterStateTest, TestReload, TestQueryTypes, TestCodecSupport, SimplePostToolTest, SampleTest, CurrencyFieldXmlFileTest, TestCollationField, IndexSchemaRuntimeFieldTest, TestPartialUpdateDeduplication, RequestHandlersTest, TestStressLucene, TestSerializedLuceneMatchVersion, TestSystemIdResolver, SolrIndexConfigTest, DOMUtilTest, PingRequestHandlerTest, StatsComponentTest, SolrPluginUtilsTest, TestShardHandlerFactory, XmlUpdateRequestHandlerTest, TestAnalyzedSuggestions, TestRandomFaceting, SignatureUpdateProcessorFactoryTest, IndexReaderFactoryTest, TestStressReorder, TestAddFieldRealTimeGet, DisMaxRequestHandlerTest, TestZkChroot, TestSuggestSpellingConverter, TestBinaryResponseWriter, TestFieldTypeCollectionResource, JSONWriterTest, NotRequiredUniqueKeyTest, PathHierarchyTokenizerFactoryTest, OpenExchangeRatesOrgProviderTest, CollectionsAPIDistributedZkTest, RequiredFieldsTest, TestElisionMultitermQuery, TestJmxMonitoredMap, OutputWriterTest, ZkSolrClientTest, BadIndexSchemaTest, UpdateParamsTest, CoreContainerCoreInitFailuresTest, TestLMJelinekMercerSimilarityFactory, TestFieldTypeResource, SpellCheckComponentTest, TestLuceneMatchVersion, TestDefaultSearchFieldResource, DateMathParserTest, URLClassifyProcessorTest, TestAtomicUpdateErrorCases, StandardRequestHandlerTest, TestWordDelimiterFilterFactory, TestHashPartitioner, TestPostingsSolrHighlighter, TestMultiCoreConfBootstrap, NoCacheHeaderTest, TestUtils, TestUpdate, NumericFieldsTest, FileUtilsTest, DebugComponentTest, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, SyncSliceTest, TestReplicationHandler, TestRecovery, ZkControllerTest, TestStressVersions, TestRangeQuery, SimpleFacetsTest, TestSolr4Spatial, SolrCmdDistributorTest, ConvertedLegacyTest, TestFunctionQuery, TestLazyCores, DirectUpdateHandlerTest, TestBadConfig, OverseerCollectionProcessorTest, TestIndexSearcher, HighlighterTest, CurrencyFieldOpenExchangeTest, TestCoreDiscovery, SpellCheckCollatorTest, CoreAdminHandlerTest, DocValuesTest, SuggesterTest, SuggesterWFSTTest, SchemaVersionSpecificBehaviorTest, QueryEqualityTest, SortByFunctionTest, TestRemoteStreaming, DistanceFunctionTest, SolrInfoMBeanTest, IndexBasedSpellCheckerTest, TestQueryUtils, DirectSolrSpellCheckerTest, TestOmitPositions, TermsComponentTest, TestIndexingPerformance, TestArbitraryIndexDir, RegexBoostProcessorTest, TestJmxIntegration, UpdateRequestProcessorFactoryTest, UniqFieldsUpdateProcessorFactoryTest, TestPHPSerializedResponseWriter, SearchHandlerTest, HighlighterConfigTest, TestPropInject, TestQuerySenderNoQuery, BadComponentTest, TestMergePolicyConfig, TestSolrDeletionPolicy2, MultiTermTest, TestBinaryField, TestSearchPerf, ExternalFileFieldSortTest, TestPhraseSuggestions, SpellPossibilityIteratorTest, SynonymTokenizerTest, TestDFRSimilarityFactory, TestNumberUtils, TestLMDirichletSimilarityFactory, TestIBSimilarityFactory, TestBM25SimilarityFactory, TestDefaultSimilarityFactory, TestPluginEnable, TimeZoneUtilsTest, ResourceLoaderTest, TestFastOutputStream, ScriptEngineTest, PluginInfoTest, ChaosMonkeyNothingIsSafeTest, PreAnalyzedFieldTest, DateFieldTest, SpellingQueryConverterTest, RAMDirectoryFactoryTest, TestSolrJ, TestLRUCache, ZkNodePropsTest, SystemInfoHandlerTest, TestRTGBase, SolrTestCaseJ4Test, ChaosMonkeyShardSplitTest]
[junit4:junit4] Completed on J1 in 295.88s, 1 test, 1 error <<< FAILURES!
[...truncated 14 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:386: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:366: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:181: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:437: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1243: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:887: There were test failures: 297 suites, 1249 tests, 1 suite-level error, 2 errors, 2 failures, 13 ignored (7 assumptions)
Total time: 62 minutes 6 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4078 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4078/
1 tests failed.
REGRESSION: org.apache.solr.cloud.ChaosMonkeyShardSplitTest.testDistribSearch
Error Message:
Server at http://127.0.0.1:22961/o_bh/e returned non ok status:500, message:Server Error
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:22961/o_bh/e returned non ok status:500, message:Server Error
at __randomizedtesting.SeedInfo.seed([BCC412BA92B002D4:3D229CA2E5EF62E8]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
at org.apache.solr.cloud.ChaosMonkeyShardSplitTest.doTest(ChaosMonkeyShardSplitTest.java:136)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 10097 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ChaosMonkeyShardSplitTest
[junit4:junit4] 2> 1697391 T3788 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /o_bh/e
[junit4:junit4] 2> 1697398 T3788 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./solrtest-ChaosMonkeyShardSplitTest-1371708601512
[junit4:junit4] 2> 1697400 T3788 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 1697401 T3789 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 1697502 T3788 oasc.ZkTestServer.run start zk server on port:34193
[junit4:junit4] 2> 1697503 T3788 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1697509 T3795 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@613ff06c name:ZooKeeperConnection Watcher:127.0.0.1:34193 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1697509 T3788 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1697510 T3788 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 1697516 T3788 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1697535 T3797 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@deabcb6 name:ZooKeeperConnection Watcher:127.0.0.1:34193/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1697536 T3788 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1697536 T3788 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 1697541 T3788 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 1697544 T3788 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 1697546 T3788 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 1697580 T3788 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 1697581 T3788 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 1697594 T3788 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 1697596 T3788 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 1697706 T3788 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 1697707 T3788 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 1697712 T3788 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 1697713 T3788 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 1697716 T3788 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 1697717 T3788 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 1697720 T3788 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 1697721 T3788 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 1697724 T3788 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 1697725 T3788 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 1697728 T3788 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 1697729 T3788 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 1697732 T3788 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 1697733 T3788 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 1698030 T3788 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1698037 T3788 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:25871
[junit4:junit4] 2> 1698039 T3788 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1698039 T3788 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1698040 T3788 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371708601855
[junit4:junit4] 2> 1698040 T3788 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371708601855/solr.xml
[junit4:junit4] 2> 1698041 T3788 oasc.CoreContainer.<init> New CoreContainer 1170945131
[junit4:junit4] 2> 1698042 T3788 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371708601855/'
[junit4:junit4] 2> 1698043 T3788 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371708601855/'
[junit4:junit4] 2> 1698198 T3788 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1698199 T3788 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1698199 T3788 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1698200 T3788 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1698201 T3788 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1698201 T3788 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1698202 T3788 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1698202 T3788 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1698203 T3788 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1698204 T3788 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1698225 T3788 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1698226 T3788 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:34193/solr
[junit4:junit4] 2> 1698227 T3788 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1698228 T3788 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1698257 T3808 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a666bba name:ZooKeeperConnection Watcher:127.0.0.1:34193 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1698258 T3788 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1698262 T3788 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1698276 T3788 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1698278 T3810 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@62e244c0 name:ZooKeeperConnection Watcher:127.0.0.1:34193/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1698279 T3788 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1698281 T3788 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 1698290 T3788 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 1698294 T3788 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 1698301 T3788 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:25871_o_bh%2Fe
[junit4:junit4] 2> 1698302 T3788 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:25871_o_bh%2Fe
[junit4:junit4] 2> 1698306 T3788 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 1698326 T3788 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 1698329 T3788 oasc.Overseer.start Overseer (id=89896294909280259-127.0.0.1:25871_o_bh%2Fe-n_0000000000) starting
[junit4:junit4] 2> 1698339 T3788 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 1698352 T3812 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 1698352 T3788 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 1698355 T3788 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 1698357 T3788 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1698360 T3811 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 1698365 T3813 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1698365 T3813 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1699863 T3811 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1699864 T3811 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:25871_o_bh%2Fe",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:25871/o_bh/e"}
[junit4:junit4] 2> 1699865 T3811 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 1699865 T3811 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1699881 T3810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1700369 T3813 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371708601855/collection1
[junit4:junit4] 2> 1700369 T3813 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 1700370 T3813 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1700370 T3813 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 1700371 T3813 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371708601855/collection1/'
[junit4:junit4] 2> 1700373 T3813 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371708601855/collection1/lib/README' to classloader
[junit4:junit4] 2> 1700373 T3813 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371708601855/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1700433 T3813 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1700501 T3813 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1700603 T3813 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1700614 T3813 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1701299 T3813 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1701304 T3813 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1701307 T3813 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1701312 T3813 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1701335 T3813 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1701336 T3813 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371708601855/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/control/data/
[junit4:junit4] 2> 1701336 T3813 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@ed331b1
[junit4:junit4] 2> 1701337 T3813 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1701337 T3813 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/control/data
[junit4:junit4] 2> 1701338 T3813 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/control/data/index/
[junit4:junit4] 2> 1701338 T3813 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1701338 T3813 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/control/data/index
[junit4:junit4] 2> 1701342 T3813 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4fbfbee4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f083e43),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1701342 T3813 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1701345 T3813 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1701346 T3813 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1701346 T3813 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1701347 T3813 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1701348 T3813 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1701348 T3813 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1701348 T3813 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1701349 T3813 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1701349 T3813 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1701362 T3813 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1701367 T3813 oass.SolrIndexSearcher.<init> Opening Searcher@60264925 main
[junit4:junit4] 2> 1701368 T3813 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/control/data/tlog
[junit4:junit4] 2> 1701369 T3813 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1701369 T3813 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1701373 T3814 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@60264925 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1701374 T3813 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1701374 T3813 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:25871/o_bh/e collection:control_collection shard:shard1
[junit4:junit4] 2> 1701375 T3813 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 1701392 T3813 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1701394 T3813 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1701395 T3813 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1701395 T3813 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:25871/o_bh/e/collection1/
[junit4:junit4] 2> 1701395 T3813 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1701395 T3813 oasc.SyncStrategy.syncToMe http://127.0.0.1:25871/o_bh/e/collection1/ has no replicas
[junit4:junit4] 2> 1701396 T3813 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:25871/o_bh/e/collection1/
[junit4:junit4] 2> 1701396 T3813 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 1702893 T3811 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1702922 T3810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1702983 T3813 oasc.ZkController.register We are http://127.0.0.1:25871/o_bh/e/collection1/ and leader is http://127.0.0.1:25871/o_bh/e/collection1/
[junit4:junit4] 2> 1702983 T3813 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:25871/o_bh/e
[junit4:junit4] 2> 1702984 T3813 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1702984 T3813 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1702984 T3813 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1702987 T3813 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1702988 T3788 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 1702989 T3788 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1702989 T3788 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1703002 T3788 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 1703004 T3788 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1703007 T3817 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@154e316c name:ZooKeeperConnection Watcher:127.0.0.1:34193/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1703007 T3788 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1703008 T3788 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1703010 T3788 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 1703199 T3788 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1703204 T3788 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:22961
[junit4:junit4] 2> 1703205 T3788 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1703206 T3788 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1703207 T3788 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371708607122
[junit4:junit4] 2> 1703207 T3788 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371708607122/solr.xml
[junit4:junit4] 2> 1703208 T3788 oasc.CoreContainer.<init> New CoreContainer 2061712758
[junit4:junit4] 2> 1703209 T3788 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371708607122/'
[junit4:junit4] 2> 1703209 T3788 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371708607122/'
[junit4:junit4] 2> 1703363 T3788 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1703364 T3788 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1703365 T3788 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1703366 T3788 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1703366 T3788 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1703367 T3788 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1703367 T3788 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1703368 T3788 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1703369 T3788 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1703369 T3788 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1703387 T3788 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1703388 T3788 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:34193/solr
[junit4:junit4] 2> 1703389 T3788 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1703390 T3788 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1703393 T3828 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@58361885 name:ZooKeeperConnection Watcher:127.0.0.1:34193 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1703394 T3788 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1703414 T3788 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1703423 T3788 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1703425 T3830 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@30efced0 name:ZooKeeperConnection Watcher:127.0.0.1:34193/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1703425 T3788 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1703435 T3788 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1704434 T3811 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1704435 T3811 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:25871_o_bh%2Fe",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:25871/o_bh/e"}
[junit4:junit4] 2> 1704463 T3788 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:22961_o_bh%2Fe
[junit4:junit4] 2> 1704482 T3788 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:22961_o_bh%2Fe
[junit4:junit4] 2> 1704483 T3810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1704483 T3817 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1704483 T3830 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1704491 T3817 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1704491 T3830 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1704498 T3810 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1704499 T3810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1704504 T3831 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1704505 T3831 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1705992 T3811 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1705993 T3811 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:22961_o_bh%2Fe",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:22961/o_bh/e"}
[junit4:junit4] 2> 1705993 T3811 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 1705993 T3811 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1706008 T3817 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1706008 T3830 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1706008 T3810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1706508 T3831 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371708607122/collection1
[junit4:junit4] 2> 1706508 T3831 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1706509 T3831 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1706509 T3831 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1706510 T3831 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371708607122/collection1/'
[junit4:junit4] 2> 1706511 T3831 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371708607122/collection1/lib/README' to classloader
[junit4:junit4] 2> 1706512 T3831 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371708607122/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1706553 T3831 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1706598 T3831 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1706699 T3831 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1706709 T3831 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1707307 T3831 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1707312 T3831 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1707315 T3831 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1707321 T3831 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1707346 T3831 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1707347 T3831 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371708607122/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty1/
[junit4:junit4] 2> 1707347 T3831 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@ed331b1
[junit4:junit4] 2> 1707348 T3831 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1707349 T3831 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty1
[junit4:junit4] 2> 1707349 T3831 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty1/index/
[junit4:junit4] 2> 1707349 T3831 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1707350 T3831 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty1/index
[junit4:junit4] 2> 1707354 T3831 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3a644592 lockFactory=org.apache.lucene.store.NativeFSLockFactory@53a0f25),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1707355 T3831 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1707359 T3831 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1707360 T3831 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1707361 T3831 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1707362 T3831 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1707363 T3831 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1707363 T3831 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1707363 T3831 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1707364 T3831 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1707365 T3831 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1707386 T3831 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1707394 T3831 oass.SolrIndexSearcher.<init> Opening Searcher@6ba0a2f1 main
[junit4:junit4] 2> 1707395 T3831 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty1/tlog
[junit4:junit4] 2> 1707396 T3831 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1707396 T3831 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1707403 T3832 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6ba0a2f1 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1707407 T3831 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1707407 T3831 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:22961/o_bh/e collection:collection1 shard:shard1
[junit4:junit4] 2> 1707408 T3831 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 1707519 T3831 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1707539 T3831 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1707540 T3831 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1707540 T3831 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:22961/o_bh/e/collection1/
[junit4:junit4] 2> 1707541 T3831 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1707541 T3831 oasc.SyncStrategy.syncToMe http://127.0.0.1:22961/o_bh/e/collection1/ has no replicas
[junit4:junit4] 2> 1707541 T3831 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:22961/o_bh/e/collection1/
[junit4:junit4] 2> 1707542 T3831 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 1709020 T3811 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1709041 T3810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1709041 T3817 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1709042 T3830 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1709074 T3831 oasc.ZkController.register We are http://127.0.0.1:22961/o_bh/e/collection1/ and leader is http://127.0.0.1:22961/o_bh/e/collection1/
[junit4:junit4] 2> 1709074 T3831 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:22961/o_bh/e
[junit4:junit4] 2> 1709075 T3831 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1709075 T3831 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1709075 T3831 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1709077 T3831 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1709079 T3788 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 1709080 T3788 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1709080 T3788 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1709398 T3788 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1709417 T3788 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:35206
[junit4:junit4] 2> 1709418 T3788 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1709419 T3788 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1709419 T3788 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371708613211
[junit4:junit4] 2> 1709420 T3788 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371708613211/solr.xml
[junit4:junit4] 2> 1709421 T3788 oasc.CoreContainer.<init> New CoreContainer 916089800
[junit4:junit4] 2> 1709421 T3788 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371708613211/'
[junit4:junit4] 2> 1709422 T3788 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371708613211/'
[junit4:junit4] 2> 1709567 T3788 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1709568 T3788 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1709569 T3788 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1709569 T3788 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1709570 T3788 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1709570 T3788 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1709571 T3788 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1709572 T3788 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1709572 T3788 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1709573 T3788 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1709590 T3788 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1709591 T3788 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:34193/solr
[junit4:junit4] 2> 1709592 T3788 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1709593 T3788 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1709597 T3844 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2f17bb5e name:ZooKeeperConnection Watcher:127.0.0.1:34193 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1709597 T3788 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1709616 T3788 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1709625 T3788 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1709627 T3846 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@377a75c2 name:ZooKeeperConnection Watcher:127.0.0.1:34193/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1709628 T3788 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1709637 T3788 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1710545 T3811 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1710546 T3811 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:22961_o_bh%2Fe",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:22961/o_bh/e"}
[junit4:junit4] 2> 1710563 T3817 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1710564 T3830 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1710563 T3846 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1710563 T3810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1710641 T3788 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35206_o_bh%2Fe
[junit4:junit4] 2> 1710644 T3788 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:35206_o_bh%2Fe
[junit4:junit4] 2> 1710646 T3846 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1710646 T3830 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1710647 T3810 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1710646 T3817 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1710647 T3810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1710647 T3830 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1710647 T3817 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1710654 T3846 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1710660 T3847 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1710660 T3847 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1712068 T3811 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1712069 T3811 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:35206_o_bh%2Fe",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35206/o_bh/e"}
[junit4:junit4] 2> 1712069 T3811 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 1712069 T3811 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 1712085 T3810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1712085 T3846 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1712085 T3830 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1712085 T3817 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1712664 T3847 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371708613211/collection1
[junit4:junit4] 2> 1712664 T3847 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1712665 T3847 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1712665 T3847 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1712667 T3847 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371708613211/collection1/'
[junit4:junit4] 2> 1712668 T3847 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371708613211/collection1/lib/README' to classloader
[junit4:junit4] 2> 1712668 T3847 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371708613211/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1712709 T3847 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1712754 T3847 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1712856 T3847 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1712865 T3847 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1713458 T3847 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1713462 T3847 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1713465 T3847 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1713470 T3847 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1713491 T3847 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1713491 T3847 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371708613211/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty2/
[junit4:junit4] 2> 1713492 T3847 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@ed331b1
[junit4:junit4] 2> 1713493 T3847 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1713493 T3847 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty2
[junit4:junit4] 2> 1713494 T3847 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty2/index/
[junit4:junit4] 2> 1713494 T3847 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1713494 T3847 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty2/index
[junit4:junit4] 2> 1713498 T3847 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3673a01 lockFactory=org.apache.lucene.store.NativeFSLockFactory@16c22487),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1713499 T3847 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1713503 T3847 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1713503 T3847 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1713504 T3847 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1713505 T3847 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1713506 T3847 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1713506 T3847 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1713506 T3847 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1713507 T3847 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1713508 T3847 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1713525 T3847 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1713533 T3847 oass.SolrIndexSearcher.<init> Opening Searcher@42318d6b main
[junit4:junit4] 2> 1713533 T3847 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty2/tlog
[junit4:junit4] 2> 1713535 T3847 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1713535 T3847 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1713541 T3848 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@42318d6b main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1713544 T3847 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1713545 T3847 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:35206/o_bh/e collection:collection1 shard:shard2
[junit4:junit4] 2> 1713546 T3847 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 1713591 T3847 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1713600 T3847 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1713600 T3847 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1713601 T3847 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:35206/o_bh/e/collection1/
[junit4:junit4] 2> 1713601 T3847 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1713601 T3847 oasc.SyncStrategy.syncToMe http://127.0.0.1:35206/o_bh/e/collection1/ has no replicas
[junit4:junit4] 2> 1713601 T3847 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:35206/o_bh/e/collection1/
[junit4:junit4] 2> 1713602 T3847 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 1715092 T3811 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1715110 T3810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1715110 T3830 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1715110 T3846 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1715110 T3817 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1715142 T3847 oasc.ZkController.register We are http://127.0.0.1:35206/o_bh/e/collection1/ and leader is http://127.0.0.1:35206/o_bh/e/collection1/
[junit4:junit4] 2> 1715142 T3847 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:35206/o_bh/e
[junit4:junit4] 2> 1715142 T3847 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1715143 T3847 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1715143 T3847 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1715145 T3847 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1715147 T3788 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 1715148 T3788 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1715149 T3788 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1715452 T3788 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1715457 T3788 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:26175
[junit4:junit4] 2> 1715458 T3788 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1715458 T3788 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1715459 T3788 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371708619281
[junit4:junit4] 2> 1715460 T3788 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371708619281/solr.xml
[junit4:junit4] 2> 1715460 T3788 oasc.CoreContainer.<init> New CoreContainer 1856980664
[junit4:junit4] 2> 1715461 T3788 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371708619281/'
[junit4:junit4] 2> 1715462 T3788 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371708619281/'
[junit4:junit4] 2> 1715636 T3788 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1715637 T3788 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1715638 T3788 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1715639 T3788 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1715639 T3788 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1715640 T3788 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1715640 T3788 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1715641 T3788 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1715642 T3788 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1715642 T3788 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1715661 T3788 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1715663 T3788 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:34193/solr
[junit4:junit4] 2> 1715663 T3788 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1715664 T3788 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1715668 T3860 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5f10726b name:ZooKeeperConnection Watcher:127.0.0.1:34193 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1715669 T3788 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1715683 T3788 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1715691 T3788 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1715694 T3862 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2a5a8b7d name:ZooKeeperConnection Watcher:127.0.0.1:34193/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1715694 T3788 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1715706 T3788 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1716623 T3811 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1716624 T3811 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:35206_o_bh%2Fe",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35206/o_bh/e"}
[junit4:junit4] 2> 1716640 T3830 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1716640 T3810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1716640 T3862 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1716640 T3817 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1716640 T3846 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1716709 T3788 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:26175_o_bh%2Fe
[junit4:junit4] 2> 1716712 T3788 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:26175_o_bh%2Fe
[junit4:junit4] 2> 1716715 T3846 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1716715 T3810 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1716715 T3817 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1716715 T3862 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1716715 T3830 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1716716 T3862 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1716716 T3817 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1716715 T3810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1716716 T3830 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1716723 T3846 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1716730 T3863 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1716731 T3863 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1718144 T3811 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1718145 T3811 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:26175_o_bh%2Fe",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:26175/o_bh/e"}
[junit4:junit4] 2> 1718145 T3811 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 1718146 T3811 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1718161 T3817 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1718161 T3846 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1718161 T3862 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1718161 T3830 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1718161 T3810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1718735 T3863 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371708619281/collection1
[junit4:junit4] 2> 1718735 T3863 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1718736 T3863 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1718736 T3863 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1718738 T3863 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371708619281/collection1/'
[junit4:junit4] 2> 1718740 T3863 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371708619281/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1718741 T3863 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371708619281/collection1/lib/README' to classloader
[junit4:junit4] 2> 1718799 T3863 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1718863 T3863 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1718964 T3863 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1718973 T3863 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1719560 T3863 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1719564 T3863 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1719567 T3863 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1719573 T3863 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1719604 T3863 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1719605 T3863 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371708619281/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty3/
[junit4:junit4] 2> 1719605 T3863 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@ed331b1
[junit4:junit4] 2> 1719606 T3863 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1719607 T3863 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty3
[junit4:junit4] 2> 1719607 T3863 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty3/index/
[junit4:junit4] 2> 1719608 T3863 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1719608 T3863 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty3/index
[junit4:junit4] 2> 1719612 T3863 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@165324f3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@63e0fab5),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1719612 T3863 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1719617 T3863 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1719617 T3863 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1719618 T3863 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1719619 T3863 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1719620 T3863 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1719620 T3863 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1719621 T3863 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1719622 T3863 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1719622 T3863 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1719639 T3863 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1719648 T3863 oass.SolrIndexSearcher.<init> Opening Searcher@153d3def main
[junit4:junit4] 2> 1719648 T3863 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty3/tlog
[junit4:junit4] 2> 1719650 T3863 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1719650 T3863 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1719658 T3864 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@153d3def main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1719661 T3863 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1719662 T3863 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:26175/o_bh/e collection:collection1 shard:shard1
[junit4:junit4] 2> 1719665 T3863 oasc.ZkController.register We are http://127.0.0.1:26175/o_bh/e/collection1/ and leader is http://127.0.0.1:22961/o_bh/e/collection1/
[junit4:junit4] 2> 1719666 T3863 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:26175/o_bh/e
[junit4:junit4] 2> 1719666 T3863 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 1719667 T3863 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C1421 name=collection1 org.apache.solr.core.SolrCore@2670bc41 url=http://127.0.0.1:26175/o_bh/e/collection1 node=127.0.0.1:26175_o_bh%2Fe C1421_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:26175_o_bh%2Fe, base_url=http://127.0.0.1:26175/o_bh/e}
[junit4:junit4] 2> 1719668 T3865 C1421 P26175 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 1719668 T3863 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1719669 T3865 C1421 P26175 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 1719670 T3865 C1421 P26175 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 1719671 T3865 C1421 P26175 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1719671 T3788 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 1719672 T3788 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1719672 T3865 C1421 P26175 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1719672 T3788 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1719697 T3820 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 1719702 T3811 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1719703 T3811 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:26175_o_bh%2Fe",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:26175/o_bh/e"}
[junit4:junit4] 2> 1719717 T3846 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1719717 T3810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1719717 T3817 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1719717 T3862 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1719717 T3830 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1719928 T3788 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1719933 T3788 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:45459
[junit4:junit4] 2> 1719934 T3788 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1719935 T3788 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1719935 T3788 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371708623807
[junit4:junit4] 2> 1719936 T3788 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371708623807/solr.xml
[junit4:junit4] 2> 1719937 T3788 oasc.CoreContainer.<init> New CoreContainer 274369780
[junit4:junit4] 2> 1719937 T3788 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371708623807/'
[junit4:junit4] 2> 1719938 T3788 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371708623807/'
[junit4:junit4] 2> 1720044 T3788 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1720045 T3788 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1720045 T3788 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1720046 T3788 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1720046 T3788 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1720047 T3788 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1720047 T3788 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1720047 T3788 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1720048 T3788 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1720048 T3788 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1720061 T3788 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1720062 T3788 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:34193/solr
[junit4:junit4] 2> 1720062 T3788 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1720063 T3788 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1720067 T3877 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2e648e8c name:ZooKeeperConnection Watcher:127.0.0.1:34193 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1720068 T3788 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1720083 T3788 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1720093 T3788 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1720095 T3879 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5e2a7248 name:ZooKeeperConnection Watcher:127.0.0.1:34193/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1720096 T3788 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1720104 T3788 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1720699 T3820 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 1720699 T3820 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=4&state=recovering&nodeName=127.0.0.1:26175_o_bh%252Fe&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1002
[junit4:junit4] 2> 1721107 T3788 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45459_o_bh%2Fe
[junit4:junit4] 2> 1721110 T3788 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45459_o_bh%2Fe
[junit4:junit4] 2> 1721112 T3846 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1721113 T3810 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1721113 T3879 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1721113 T3862 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1721113 T3830 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1721114 T3846 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1721113 T3817 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1721114 T3830 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1721114 T3862 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1721113 T3810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1721115 T3817 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1721134 T3880 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1721134 T3880 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1721221 T3811 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1721222 T3811 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:45459_o_bh%2Fe",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:45459/o_bh/e"}
[junit4:junit4] 2> 1721223 T3811 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 1721223 T3811 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 1721236 T3846 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1721236 T3830 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1721236 T3817 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1721236 T3879 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1721236 T3810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1721236 T3862 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1722137 T3880 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371708623807/collection1
[junit4:junit4] 2> 1722137 T3880 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1722138 T3880 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1722138 T3880 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1722140 T3880 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371708623807/collection1/'
[junit4:junit4] 2> 1722142 T3880 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371708623807/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1722142 T3880 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371708623807/collection1/lib/README' to classloader
[junit4:junit4] 2> 1722204 T3880 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1722277 T3880 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1722378 T3880 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1722384 T3880 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> ASYNC NEW_CORE C1422 name=collection1 org.apache.solr.core.SolrCore@2670bc41 url=http://127.0.0.1:26175/o_bh/e/collection1 node=127.0.0.1:26175_o_bh%2Fe C1422_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:26175_o_bh%2Fe, base_url=http://127.0.0.1:26175/o_bh/e}
[junit4:junit4] 2> 1722701 T3865 C1422 P26175 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:22961/o_bh/e/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 1722701 T3865 C1422 P26175 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:26175/o_bh/e START replicas=[http://127.0.0.1:22961/o_bh/e/collection1/] nUpdates=100
[junit4:junit4] 2> 1722702 T3865 C1422 P26175 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 1722702 T3865 C1422 P26175 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 1722702 T3865 C1422 P26175 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 1722702 T3865 C1422 P26175 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 1722702 T3865 C1422 P26175 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 1722703 T3865 C1422 P26175 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:22961/o_bh/e/collection1/. core=collection1
[junit4:junit4] 2> 1722703 T3865 C1422 P26175 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C1423 name=collection1 org.apache.solr.core.SolrCore@2d1897fb url=http://127.0.0.1:22961/o_bh/e/collection1 node=127.0.0.1:22961_o_bh%2Fe C1423_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:22961_o_bh%2Fe, base_url=http://127.0.0.1:22961/o_bh/e, leader=true}
[junit4:junit4] 2> 1722710 T3821 C1423 P22961 oasc.SolrCore.execute [collection1] webapp=/o_bh/e path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=2
[junit4:junit4] 2> 1722714 T3822 C1423 P22961 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1722718 T3822 C1423 P22961 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3a644592 lockFactory=org.apache.lucene.store.NativeFSLockFactory@53a0f25),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1722720 T3822 C1423 P22961 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1722721 T3822 C1423 P22961 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3a644592 lockFactory=org.apache.lucene.store.NativeFSLockFactory@53a0f25),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3a644592 lockFactory=org.apache.lucene.store.NativeFSLockFactory@53a0f25),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1722721 T3822 C1423 P22961 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1722722 T3822 C1423 P22961 oass.SolrIndexSearcher.<init> Opening Searcher@4f33d784 realtime
[junit4:junit4] 2> 1722723 T3822 C1423 P22961 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1722723 T3822 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4] 2> 1722724 T3865 C1422 P26175 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1722725 T3865 C1422 P26175 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 1722727 T3823 C1423 P22961 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1722728 T3823 C1423 P22961 oasc.SolrCore.execute [collection1] webapp=/o_bh/e path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2
[junit4:junit4] 2> 1722729 T3865 C1422 P26175 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 1722729 T3865 C1422 P26175 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 1722729 T3865 C1422 P26175 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 1722732 T3823 C1423 P22961 oasc.SolrCore.execute [collection1] webapp=/o_bh/e path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 1722733 T3865 C1422 P26175 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 1722734 T3865 C1422 P26175 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty3/index.20130620021026845
[junit4:junit4] 2> 1722735 T3865 C1422 P26175 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@5f3bf202 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7cd1982) fullCopy=false
[junit4:junit4] 2> 1722737 T3823 C1423 P22961 oasc.SolrCore.execute [collection1] webapp=/o_bh/e path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 1722739 T3865 C1422 P26175 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 1722740 T3865 C1422 P26175 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 1722741 T3865 C1422 P26175 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 1722743 T3865 C1422 P26175 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@165324f3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@63e0fab5),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@165324f3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@63e0fab5),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1722744 T3865 C1422 P26175 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1722744 T3865 C1422 P26175 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 1722744 T3865 C1422 P26175 oass.SolrIndexSearcher.<init> Opening Searcher@22ffc604 main
[junit4:junit4] 2> 1722746 T3864 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@22ffc604 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1722746 T3865 C1422 P26175 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty3/index.20130620021026845 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty3/index.20130620021026845;done=true>>]
[junit4:junit4] 2> 1722747 T3865 C1422 P26175 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty3/index.20130620021026845
[junit4:junit4] 2> 1722747 T3865 C1422 P26175 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty3/index.20130620021026845
[junit4:junit4] 2> 1722747 T3865 C1422 P26175 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 1722748 T3865 C1422 P26175 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 1722748 T3865 C1422 P26175 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1722748 T3865 C1422 P26175 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1722750 T3865 C1422 P26175 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 1722855 T3880 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1722858 T3880 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1722860 T3880 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1722865 T3880 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1722892 T3880 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1722892 T3880 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371708623807/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty4/
[junit4:junit4] 2> 1722893 T3880 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@ed331b1
[junit4:junit4] 2> 1722893 T3880 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1722894 T3880 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty4
[junit4:junit4] 2> 1722894 T3880 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty4/index/
[junit4:junit4] 2> 1722894 T3880 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1722895 T3880 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty4/index
[junit4:junit4] 2> 1722898 T3880 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@49131556 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b3409dc),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1722898 T3880 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1722901 T3880 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1722902 T3880 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1722902 T3880 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1722903 T3880 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1722903 T3880 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1722904 T3880 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1722905 T3880 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1722906 T3880 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1722906 T3880 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1722918 T3880 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1722926 T3880 oass.SolrIndexSearcher.<init> Opening Searcher@50f96063 main
[junit4:junit4] 2> 1722926 T3880 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty4/tlog
[junit4:junit4] 2> 1722927 T3880 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1722928 T3880 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1722931 T3883 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@50f96063 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1722933 T3880 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1722934 T3880 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:45459/o_bh/e collection:collection1 shard:shard2
[junit4:junit4] 2> 1722936 T3880 oasc.ZkController.register We are http://127.0.0.1:45459/o_bh/e/collection1/ and leader is http://127.0.0.1:35206/o_bh/e/collection1/
[junit4:junit4] 2> 1722937 T3880 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:45459/o_bh/e
[junit4:junit4] 2> 1722937 T3880 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 1722937 T3880 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C1424 name=collection1 org.apache.solr.core.SolrCore@3198de41 url=http://127.0.0.1:45459/o_bh/e/collection1 node=127.0.0.1:45459_o_bh%2Fe C1424_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:45459_o_bh%2Fe, base_url=http://127.0.0.1:45459/o_bh/e}
[junit4:junit4] 2> 1722937 T3884 C1424 P45459 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 1722938 T3880 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1722938 T3884 C1424 P45459 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 1722939 T3884 C1424 P45459 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 1722939 T3884 C1424 P45459 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1722939 T3788 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 1722940 T3788 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1722940 T3884 C1424 P45459 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1722941 T3788 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1722960 T3788 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1722960 T3836 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 1722961 T3788 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 1722962 T3788 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 1722962 T3788 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1723964 T3788 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1724243 T3811 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1724244 T3811 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:26175_o_bh%2Fe",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:26175/o_bh/e"}
[junit4:junit4] 2> 1724277 T3811 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:45459_o_bh%2Fe",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:45459/o_bh/e"}
[junit4:junit4] 2> 1724355 T3862 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1724355 T3879 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1724355 T3830 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1724355 T3846 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1724355 T3810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1724355 T3817 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1724963 T3836 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 1724964 T3836 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=5&state=recovering&nodeName=127.0.0.1:45459_o_bh%252Fe&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2004
[junit4:junit4] 2> 1724967 T3788 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1725970 T3788 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C1424_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:45459_o_bh%2Fe, base_url=http://127.0.0.1:45459/o_bh/e}
[junit4:junit4] 2> 1726965 T3884 C1424 P45459 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:35206/o_bh/e/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 1726965 T3884 C1424 P45459 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:45459/o_bh/e START replicas=[http://127.0.0.1:35206/o_bh/e/collection1/] nUpdates=100
[junit4:junit4] 2> 1726966 T3884 C1424 P45459 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 1726966 T3884 C1424 P45459 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 1726966 T3884 C1424 P45459 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 1726966 T3884 C1424 P45459 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 1726966 T3884 C1424 P45459 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 1726967 T3884 C1424 P45459 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:35206/o_bh/e/collection1/. core=collection1
[junit4:junit4] 2> 1726967 T3884 C1424 P45459 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C1425 name=collection1 org.apache.solr.core.SolrCore@78585fbf url=http://127.0.0.1:35206/o_bh/e/collection1 node=127.0.0.1:35206_o_bh%2Fe C1425_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:35206_o_bh%2Fe, base_url=http://127.0.0.1:35206/o_bh/e, leader=true}
[junit4:junit4] 2> 1726973 T3837 C1425 P35206 oasc.SolrCore.execute [collection1] webapp=/o_bh/e path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 1726974 T3788 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1726977 T3838 C1425 P35206 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1726980 T3838 C1425 P35206 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3673a01 lockFactory=org.apache.lucene.store.NativeFSLockFactory@16c22487),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1726980 T3838 C1425 P35206 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1726981 T3838 C1425 P35206 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3673a01 lockFactory=org.apache.lucene.store.NativeFSLockFactory@16c22487),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3673a01 lockFactory=org.apache.lucene.store.NativeFSLockFactory@16c22487),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1726981 T3838 C1425 P35206 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1726982 T3838 C1425 P35206 oass.SolrIndexSearcher.<init> Opening Searcher@293c15f5 realtime
[junit4:junit4] 2> 1726982 T3838 C1425 P35206 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1726982 T3838 C1425 P35206 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> 1726983 T3884 C1424 P45459 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1726984 T3884 C1424 P45459 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 1726985 T3839 C1425 P35206 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1726986 T3839 C1425 P35206 oasc.SolrCore.execute [collection1] webapp=/o_bh/e path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 1726986 T3884 C1424 P45459 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 1726987 T3884 C1424 P45459 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 1726987 T3884 C1424 P45459 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 1726989 T3839 C1425 P35206 oasc.SolrCore.execute [collection1] webapp=/o_bh/e path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 1726990 T3884 C1424 P45459 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 1726991 T3884 C1424 P45459 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty4/index.20130620021031102
[junit4:junit4] 2> 1726991 T3884 C1424 P45459 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@4c9a081c lockFactory=org.apache.lucene.store.NativeFSLockFactory@71e92179) fullCopy=false
[junit4:junit4] 2> 1726994 T3839 C1425 P35206 oasc.SolrCore.execute [collection1] webapp=/o_bh/e path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 1726995 T3884 C1424 P45459 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 1726996 T3884 C1424 P45459 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 1726996 T3884 C1424 P45459 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 1726998 T3884 C1424 P45459 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@49131556 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b3409dc),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@49131556 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b3409dc),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1726998 T3884 C1424 P45459 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1726998 T3884 C1424 P45459 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 1726999 T3884 C1424 P45459 oass.SolrIndexSearcher.<init> Opening Searcher@2e709c0c main
[junit4:junit4] 2> 1727000 T3883 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2e709c0c main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1727000 T3884 C1424 P45459 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty4/index.20130620021031102 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty4/index.20130620021031102;done=true>>]
[junit4:junit4] 2> 1727000 T3884 C1424 P45459 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty4/index.20130620021031102
[junit4:junit4] 2> 1727001 T3884 C1424 P45459 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty4/index.20130620021031102
[junit4:junit4] 2> 1727001 T3884 C1424 P45459 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 1727001 T3884 C1424 P45459 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 1727001 T3884 C1424 P45459 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1727001 T3884 C1424 P45459 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1727003 T3884 C1424 P45459 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 1727366 T3811 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1727367 T3811 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:45459_o_bh%2Fe",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:45459/o_bh/e"}
[junit4:junit4] 2> 1727428 T3862 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1727428 T3879 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1727428 T3817 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1727428 T3830 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1727428 T3810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1727428 T3846 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1727976 T3788 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1727978 T3788 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C1426 name=collection1 org.apache.solr.core.SolrCore@1ca6ebb9 url=http://127.0.0.1:25871/o_bh/e/collection1 node=127.0.0.1:25871_o_bh%2Fe C1426_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:25871_o_bh%2Fe, base_url=http://127.0.0.1:25871/o_bh/e, leader=true}
[junit4:junit4] 2> 1727987 T3800 C1426 P25871 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1727993 T3800 C1426 P25871 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4fbfbee4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f083e43),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1727994 T3800 C1426 P25871 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1727995 T3800 C1426 P25871 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4fbfbee4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f083e43),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4fbfbee4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f083e43),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1727996 T3800 C1426 P25871 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1727997 T3800 C1426 P25871 oass.SolrIndexSearcher.<init> Opening Searcher@1f813456 main
[junit4:junit4] 2> 1727997 T3800 C1426 P25871 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1727998 T3814 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1f813456 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1727999 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 12
[junit4:junit4] 2> C1424_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:45459_o_bh%2Fe, base_url=http://127.0.0.1:45459/o_bh/e}
[junit4:junit4] 2> 1728021 T3869 C1424 P45459 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:22961/o_bh/e/collection1/, StdNode: http://127.0.0.1:26175/o_bh/e/collection1/, StdNode: http://127.0.0.1:35206/o_bh/e/collection1/, StdNode: http://127.0.0.1:45459/o_bh/e/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 1728026 T3840 C1425 P35206 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1728026 T3824 C1423 P22961 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1728026 T3870 C1424 P45459 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1728030 T3824 C1423 P22961 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3a644592 lockFactory=org.apache.lucene.store.NativeFSLockFactory@53a0f25),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3a644592 lockFactory=org.apache.lucene.store.NativeFSLockFactory@53a0f25),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1728028 T3840 C1425 P35206 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3673a01 lockFactory=org.apache.lucene.store.NativeFSLockFactory@16c22487),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3673a01 lockFactory=org.apache.lucene.store.NativeFSLockFactory@16c22487),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1728032 T3824 C1423 P22961 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1728031 T3870 C1424 P45459 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@49131556 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b3409dc),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@49131556 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b3409dc),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1728035 T3824 C1423 P22961 oass.SolrIndexSearcher.<init> Opening Searcher@2bf5d50e main
[junit4:junit4] 2> 1728034 T3840 C1425 P35206 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1728036 T3824 C1423 P22961 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1728035 T3870 C1424 P45459 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1728037 T3832 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2bf5d50e main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1728037 T3840 C1425 P35206 oass.SolrIndexSearcher.<init> Opening Searcher@142f0c0e main
[junit4:junit4] 2> 1728039 T3824 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 13
[junit4:junit4] 2> 1728038 T3870 C1424 P45459 oass.SolrIndexSearcher.<init> Opening Searcher@253cdf6d main
[junit4:junit4] 2> C1422_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:26175_o_bh%2Fe, base_url=http://127.0.0.1:26175/o_bh/e}
[junit4:junit4] 2> 1728038 T3852 C1422 P26175 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1728040 T3870 C1424 P45459 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1728040 T3848 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@142f0c0e main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1728039 T3840 C1425 P35206 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1728042 T3852 C1422 P26175 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@165324f3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@63e0fab5),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@165324f3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@63e0fab5),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1728042 T3883 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@253cdf6d main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1728044 T3852 C1422 P26175 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1728043 T3840 C1425 P35206 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 17
[junit4:junit4] 2> 1728045 T3852 C1422 P26175 oass.SolrIndexSearcher.<init> Opening Searcher@5f73411a main
[junit4:junit4] 2> 1728044 T3870 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 18
[junit4:junit4] 2> 1728046 T3852 C1422 P26175 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1728047 T3864 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5f73411a main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1728048 T3852 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4] 2> 1728049 T3869 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 28
[junit4:junit4] 2> 1728050 T3788 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1728053 T3825 C1423 P22961 oasc.SolrCore.execute [collection1] webapp=/o_bh/e path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 1728056 T3853 C1422 P26175 oasc.SolrCore.execute [collection1] webapp=/o_bh/e path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 1728058 T3841 C1425 P35206 oasc.SolrCore.execute [collection1] webapp=/o_bh/e path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 1728061 T3871 C1424 P45459 oasc.SolrCore.execute [collection1] webapp=/o_bh/e path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 1730067 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1438340752791830528)} 0 2
[junit4:junit4] 2> 1730078 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={update.distrib=FROMLEADER&_version_=-1438340752796024832&update.from=http://127.0.0.1:22961/o_bh/e/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438340752796024832)} 0 2
[junit4:junit4] 2> 1730083 T3872 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={update.distrib=FROMLEADER&_version_=-1438340752802316288&update.from=http://127.0.0.1:35206/o_bh/e/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438340752802316288)} 0 2
[junit4:junit4] 2> 1730084 T3842 C1425 P35206 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1438340752802316288)} 0 8
[junit4:junit4] 2> 1730084 T3826 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1438340752796024832)} 0 14
[junit4:junit4] 2> 1730088 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[0 (1438340752814899200)]} 0 1
[junit4:junit4] 2> 1730098 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438340752821190656)]} 0 1
[junit4:junit4] 2> 1730099 T3824 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:45459/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[0 (1438340752821190656)]} 0 5
[junit4:junit4] 2> 1730100 T3869 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[0]} 0 9
[junit4:junit4] 2> 1730104 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[1 (1438340752830627840)]} 0 1
[junit4:junit4] 2> 1730111 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438340752834822144)]} 0 1
[junit4:junit4] 2> 1730112 T3826 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[1 (1438340752834822144)]} 0 5
[junit4:junit4] 2> 1730116 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[2 (1438340752843210752)]} 0 1
[junit4:junit4] 2> 1730125 T3872 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:35206/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1438340752849502208)]} 0 1
[junit4:junit4] 2> 1730126 T3842 C1425 P35206 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[2 (1438340752849502208)]} 0 5
[junit4:junit4] 2> 1730127 T3826 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[2]} 0 8
[junit4:junit4] 2> 1730131 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[3 (1438340752859987968)]} 0 1
[junit4:junit4] 2> 1730141 T3872 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:35206/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1438340752866279424)]} 0 1
[junit4:junit4] 2> 1730142 T3840 C1425 P35206 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:45459/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[3 (1438340752866279424)]} 0 5
[junit4:junit4] 2> 1730143 T3869 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[3]} 0 9
[junit4:junit4] 2> 1730146 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[4 (1438340752874668032)]} 0 1
[junit4:junit4] 2> 1730153 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1438340752878862336)]} 0 1
[junit4:junit4] 2> 1730154 T3826 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[4 (1438340752878862336)]} 0 5
[junit4:junit4] 2> 1730157 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[5 (1438340752886202368)]} 0 1
[junit4:junit4] 2> 1730165 T3872 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:35206/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[5 (1438340752892493824)]} 0 1
[junit4:junit4] 2> 1730166 T3842 C1425 P35206 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[5 (1438340752892493824)]} 0 5
[junit4:junit4] 2> 1730167 T3826 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[5]} 0 8
[junit4:junit4] 2> 1730171 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[6 (1438340752900882432)]} 0 1
[junit4:junit4] 2> 1730181 T3872 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:35206/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[6 (1438340752908222464)]} 0 1
[junit4:junit4] 2> 1730182 T3842 C1425 P35206 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[6 (1438340752908222464)]} 0 5
[junit4:junit4] 2> 1730183 T3826 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[6]} 0 9
[junit4:junit4] 2> 1730187 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[7 (1438340752917659648)]} 0 1
[junit4:junit4] 2> 1730197 T3872 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:35206/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[7 (1438340752924999680)]} 0 1
[junit4:junit4] 2> 1730198 T3842 C1425 P35206 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[7 (1438340752924999680)]} 0 5
[junit4:junit4] 2> 1730199 T3826 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[7]} 0 9
[junit4:junit4] 2> 1730203 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[8 (1438340752934436864)]} 0 1
[junit4:junit4] 2> 1730210 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1438340752938631168)]} 0 1
[junit4:junit4] 2> 1730211 T3826 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[8 (1438340752938631168)]} 0 5
[junit4:junit4] 2> 1730215 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[9 (1438340752947019776)]} 0 1
[junit4:junit4] 2> 1730224 T3872 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:35206/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[9 (1438340752953311232)]} 0 1
[junit4:junit4] 2> 1730225 T3836 C1425 P35206 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:26175/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[9 (1438340752953311232)]} 0 5
[junit4:junit4] 2> 1730226 T3855 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[9]} 0 9
[junit4:junit4] 2> 1730230 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[10 (1438340752962748416)]} 0 1
[junit4:junit4] 2> 1730237 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1438340752966942720)]} 0 1
[junit4:junit4] 2> 1730238 T3826 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[10 (1438340752966942720)]} 0 5
[junit4:junit4] 2> 1730242 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[11 (1438340752975331328)]} 0 1
[junit4:junit4] 2> 1730249 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[11 (1438340752979525632)]} 0 1
[junit4:junit4] 2> 1730250 T3820 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:35206/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[11 (1438340752979525632)]} 0 5
[junit4:junit4] 2> 1730250 T3838 C1425 P35206 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[11]} 0 6
[junit4:junit4] 2> 1730254 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[12 (1438340752987914240)]} 0 1
[junit4:junit4] 2> 1730261 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[12 (1438340752992108544)]} 0 1
[junit4:junit4] 2> 1730262 T3826 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[12 (1438340752992108544)]} 0 5
[junit4:junit4] 2> 1730266 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[13 (1438340753000497152)]} 0 1
[junit4:junit4] 2> 1730275 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[13 (1438340753006788608)]} 0 1
[junit4:junit4] 2> 1730276 T3822 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:26175/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[13 (1438340753006788608)]} 0 5
[junit4:junit4] 2> 1730277 T3855 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[13]} 0 8
[junit4:junit4] 2> 1730281 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[14 (1438340753016225792)]} 0 1
[junit4:junit4] 2> 1730291 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[14 (1438340753023565824)]} 0 1
[junit4:junit4] 2> 1730292 T3824 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:45459/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[14 (1438340753023565824)]} 0 5
[junit4:junit4] 2> 1730292 T3869 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[14]} 0 8
[junit4:junit4] 2> 1730296 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[15 (1438340753031954432)]} 0 1
[junit4:junit4] 2> 1730305 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[15 (1438340753038245888)]} 0 1
[junit4:junit4] 2> 1730305 T3824 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:45459/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[15 (1438340753038245888)]} 0 4
[junit4:junit4] 2> 1730306 T3869 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[15]} 0 7
[junit4:junit4] 2> 1730310 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[16 (1438340753046634496)]} 0 1
[junit4:junit4] 2> 1730316 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[16 (1438340753050828800)]} 0 0
[junit4:junit4] 2> 1730317 T3826 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[16 (1438340753050828800)]} 0 4
[junit4:junit4] 2> 1730321 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[17 (1438340753058168832)]} 0 1
[junit4:junit4] 2> 1730327 T3872 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:35206/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[17 (1438340753061314560)]} 0 1
[junit4:junit4] 2> 1730328 T3838 C1425 P35206 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[17 (1438340753061314560)]} 0 5
[junit4:junit4] 2> 1730331 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[18 (1438340753068654592)]} 0 1
[junit4:junit4] 2> 1730340 T3872 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:35206/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[18 (1438340753074946048)]} 0 1
[junit4:junit4] 2> 1730340 T3840 C1425 P35206 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:45459/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[18 (1438340753074946048)]} 0 4
[junit4:junit4] 2> 1730341 T3869 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[18]} 0 7
[junit4:junit4] 2> 1730345 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[19 (1438340753083334656)]} 0 1
[junit4:junit4] 2> 1730351 T3872 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:35206/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[19 (1438340753087528960)]} 0 0
[junit4:junit4] 2> 1730352 T3838 C1425 P35206 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[19 (1438340753087528960)]} 0 4
[junit4:junit4] 2> 1730356 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[20 (1438340753094868992)]} 0 1
[junit4:junit4] 2> 1730363 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[20 (1438340753099063296)]} 0 0
[junit4:junit4] 2> 1730363 T3826 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[20 (1438340753099063296)]} 0 4
[junit4:junit4] 2> 1730366 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[21 (1438340753106403328)]} 0 0
[junit4:junit4] 2> 1730375 T3872 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:35206/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[21 (1438340753111646208)]} 0 1
[junit4:junit4] 2> 1730376 T3840 C1425 P35206 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:45459/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[21 (1438340753111646208)]} 0 5
[junit4:junit4] 2> 1730376 T3869 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[21]} 0 7
[junit4:junit4] 2> 1730380 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[22 (1438340753120034816)]} 0 1
[junit4:junit4] 2> 1730391 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[22 (1438340753128423424)]} 0 1
[junit4:junit4] 2> 1730392 T3824 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:45459/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[22 (1438340753128423424)]} 0 5
[junit4:junit4] 2> 1730393 T3869 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[22]} 0 10
[junit4:junit4] 2> 1730397 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[23 (1438340753137860608)]} 0 1
[junit4:junit4] 2> 1730413 T3872 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:35206/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[23 (1438340753151492096)]} 0 1
[junit4:junit4] 2> 1730415 T3840 C1425 P35206 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:45459/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[23 (1438340753151492096)]} 0 7
[junit4:junit4] 2> 1730416 T3869 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[23]} 0 16
[junit4:junit4] 2> 1730420 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[24 (1438340753161977856)]} 0 1
[junit4:junit4] 2> 1730426 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[24 (1438340753165123584)]} 0 1
[junit4:junit4] 2> 1730427 T3826 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[24 (1438340753165123584)]} 0 5
[junit4:junit4] 2> 1730431 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[25 (1438340753173512192)]} 0 1
[junit4:junit4] 2> 1730440 T3872 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:35206/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[25 (1438340753179803648)]} 0 1
[junit4:junit4] 2> 1730441 T3842 C1425 P35206 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[25 (1438340753179803648)]} 0 5
[junit4:junit4] 2> 1730442 T3826 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[25]} 0 9
[junit4:junit4] 2> 1730446 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[26 (1438340753189240832)]} 0 1
[junit4:junit4] 2> 1730453 T3872 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:35206/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[26 (1438340753193435136)]} 0 1
[junit4:junit4] 2> 1730454 T3838 C1425 P35206 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[26 (1438340753193435136)]} 0 5
[junit4:junit4] 2> 1730458 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[27 (1438340753201823744)]} 0 1
[junit4:junit4] 2> 1730468 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[27 (1438340753209163776)]} 0 1
[junit4:junit4] 2> 1730469 T3824 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:45459/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[27 (1438340753209163776)]} 0 5
[junit4:junit4] 2> 1730470 T3869 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[27]} 0 9
[junit4:junit4] 2> 1730474 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[28 (1438340753218600960)]} 0 1
[junit4:junit4] 2> 1730483 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[28 (1438340753224892416)]} 0 1
[junit4:junit4] 2> 1730484 T3822 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:26175/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[28 (1438340753224892416)]} 0 5
[junit4:junit4] 2> 1730484 T3855 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[28]} 0 7
[junit4:junit4] 2> 1730488 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[29 (1438340753233281024)]} 0 1
[junit4:junit4] 2> 1730496 T3872 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:35206/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[29 (1438340753238523904)]} 0 1
[junit4:junit4] 2> 1730497 T3840 C1425 P35206 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:45459/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[29 (1438340753238523904)]} 0 5
[junit4:junit4] 2> 1730497 T3869 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[29]} 0 7
[junit4:junit4] 2> 1730503 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[30 (1438340753246912512)]} 0 3
[junit4:junit4] 2> 1730512 T3872 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:35206/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[30 (1438340753256349696)]} 0 0
[junit4:junit4] 2> 1730513 T3840 C1425 P35206 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:45459/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[30 (1438340753256349696)]} 0 4
[junit4:junit4] 2> 1730514 T3869 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[30]} 0 8
[junit4:junit4] 2> 1730518 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[31 (1438340753264738304)]} 0 1
[junit4:junit4] 2> 1730526 T3872 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:35206/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[31 (1438340753269981184)]} 0 1
[junit4:junit4] 2> 1730527 T3836 C1425 P35206 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:26175/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[31 (1438340753269981184)]} 0 5
[junit4:junit4] 2> 1730528 T3855 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[31]} 0 8
[junit4:junit4] 2> 1730532 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[32 (1438340753279418368)]} 0 1
[junit4:junit4] 2> 1730542 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[32 (1438340753287806976)]} 0 0
[junit4:junit4] 2> 1730544 T3824 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:45459/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[32 (1438340753287806976)]} 0 5
[junit4:junit4] 2> 1730545 T3869 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[32]} 0 9
[junit4:junit4] 2> 1730549 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[33 (1438340753297244160)]} 0 1
[junit4:junit4] 2> 1730557 T3872 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:35206/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[33 (1438340753302487040)]} 0 1
[junit4:junit4] 2> 1730558 T3842 C1425 P35206 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[33 (1438340753302487040)]} 0 5
[junit4:junit4] 2> 1730559 T3826 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[33]} 0 8
[junit4:junit4] 2> 1730563 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[34 (1438340753311924224)]} 0 1
[junit4:junit4] 2> 1730569 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[34 (1438340753315069952)]} 0 1
[junit4:junit4] 2> 1730570 T3826 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[34 (1438340753315069952)]} 0 5
[junit4:junit4] 2> 1730574 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[35 (1438340753323458560)]} 0 1
[junit4:junit4] 2> 1730583 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[35 (1438340753330798592)]} 0 0
[junit4:junit4] 2> 1730584 T3820 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:35206/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[35 (1438340753330798592)]} 0 4
[junit4:junit4] 2> 1730585 T3838 C1425 P35206 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[35]} 0 8
[junit4:junit4] 2> 1730589 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[36 (1438340753339187200)]} 0 1
[junit4:junit4] 2> 1730596 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[36 (1438340753344430080)]} 0 0
[junit4:junit4] 2> 1730597 T3820 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:35206/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[36 (1438340753344430080)]} 0 4
[junit4:junit4] 2> 1730598 T3838 C1425 P35206 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[36]} 0 7
[junit4:junit4] 2> 1730601 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[37 (1438340753352818688)]} 0 0
[junit4:junit4] 2> 1730610 T3872 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:35206/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[37 (1438340753358061568)]} 0 1
[junit4:junit4] 2> 1730611 T3836 C1425 P35206 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:26175/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[37 (1438340753358061568)]} 0 5
[junit4:junit4] 2> 1730611 T3855 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[37]} 0 7
[junit4:junit4] 2> 1730615 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[38 (1438340753366450176)]} 0 1
[junit4:junit4] 2> 1730622 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[38 (1438340753370644480)]} 0 1
[junit4:junit4] 2> 1730623 T3826 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[38 (1438340753370644480)]} 0 5
[junit4:junit4] 2> 1730627 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[39 (1438340753379033088)]} 0 1
[junit4:junit4] 2> 1730636 T3872 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:35206/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[39 (1438340753385324544)]} 0 1
[junit4:junit4] 2> 1730637 T3842 C1425 P35206 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[39 (1438340753385324544)]} 0 5
[junit4:junit4] 2> 1730637 T3826 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[39]} 0 7
[junit4:junit4] 2> 1730641 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[40 (1438340753393713152)]} 0 1
[junit4:junit4] 2> 1730650 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[40 (1438340753401053184)]} 0 0
[junit4:junit4] 2> 1730652 T3824 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:45459/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[40 (1438340753401053184)]} 0 5
[junit4:junit4] 2> 1730652 T3869 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[40]} 0 8
[junit4:junit4] 2> 1730656 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[41 (1438340753409441792)]} 0 1
[junit4:junit4] 2> 1730666 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[41 (1438340753416781824)]} 0 1
[junit4:junit4] 2> 1730667 T3824 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:45459/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[41 (1438340753416781824)]} 0 5
[junit4:junit4] 2> 1730668 T3869 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[41]} 0 9
[junit4:junit4] 2> 1730672 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[42 (1438340753426219008)]} 0 1
[junit4:junit4] 2> 1730679 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[42 (1438340753430413312)]} 0 1
[junit4:junit4] 2> 1730682 T3826 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[42 (1438340753430413312)]} 0 7
[junit4:junit4] 2> 1730687 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[43 (1438340753441947648)]} 0 1
[junit4:junit4] 2> 1730694 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[43 (1438340753446141952)]} 0 1
[junit4:junit4] 2> 1730695 T3826 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[43 (1438340753446141952)]} 0 5
[junit4:junit4] 2> 1730699 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[44 (1438340753454530560)]} 0 1
[junit4:junit4] 2> 1730709 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[44 (1438340753461870592)]} 0 1
[junit4:junit4] 2> 1730710 T3824 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:45459/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[44 (1438340753461870592)]} 0 5
[junit4:junit4] 2> 1730711 T3869 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[44]} 0 9
[junit4:junit4] 2> 1730715 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[45 (1438340753471307776)]} 0 1
[junit4:junit4] 2> 1730722 T3872 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:35206/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[45 (1438340753475502080)]} 0 1
[junit4:junit4] 2> 1730723 T3838 C1425 P35206 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[45 (1438340753475502080)]} 0 5
[junit4:junit4] 2> 1730727 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[46 (1438340753483890688)]} 0 1
[junit4:junit4] 2> 1730737 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[46 (1438340753491230720)]} 0 1
[junit4:junit4] 2> 1730738 T3822 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:26175/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[46 (1438340753491230720)]} 0 5
[junit4:junit4] 2> 1730739 T3855 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[46]} 0 9
[junit4:junit4] 2> 1730743 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[47 (1438340753500667904)]} 0 1
[junit4:junit4] 2> 1730750 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[47 (1438340753504862208)]} 0 1
[junit4:junit4] 2> 1730751 T3826 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[47 (1438340753504862208)]} 0 5
[junit4:junit4] 2> 1730755 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[48 (1438340753513250816)]} 0 1
[junit4:junit4] 2> 1730765 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[48 (1438340753520590848)]} 0 1
[junit4:junit4] 2> 1730766 T3822 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:26175/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[48 (1438340753520590848)]} 0 5
[junit4:junit4] 2> 1730767 T3855 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[48]} 0 9
[junit4:junit4] 2> 1730771 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[49 (1438340753530028032)]} 0 1
[junit4:junit4] 2> 1730781 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[49 (1438340753537368064)]} 0 1
[junit4:junit4] 2> 1730782 T3824 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:45459/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[49 (1438340753537368064)]} 0 5
[junit4:junit4] 2> 1730783 T3869 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[49]} 0 9
[junit4:junit4] 2> 1730786 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[50 (1438340753545756672)]} 0 1
[junit4:junit4] 2> 1730795 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[50 (1438340753552048128)]} 0 1
[junit4:junit4] 2> 1730796 T3820 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:35206/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[50 (1438340753552048128)]} 0 5
[junit4:junit4] 2> 1730797 T3838 C1425 P35206 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[50]} 0 8
[junit4:junit4] 2> 1730799 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[51 (1438340753560436736)]} 0 0
[junit4:junit4] 2> 1730807 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[51 (1438340753565679616)]} 0 0
[junit4:junit4] 2> 1730808 T3822 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:26175/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[51 (1438340753565679616)]} 0 4
[junit4:junit4] 2> 1730809 T3855 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[51]} 0 7
[junit4:junit4] 2> 1730812 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[52 (1438340753574068224)]} 0 0
[junit4:junit4] 2> 1730819 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[52 (1438340753577213952)]} 0 1
[junit4:junit4] 2> 1730820 T3826 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[52 (1438340753577213952)]} 0 5
[junit4:junit4] 2> 1730824 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[53 (1438340753585602560)]} 0 1
[junit4:junit4] 2> 1730833 T3872 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:35206/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[53 (1438340753591894016)]} 0 1
[junit4:junit4] 2> 1730834 T3840 C1425 P35206 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:45459/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[53 (1438340753591894016)]} 0 5
[junit4:junit4] 2> 1730835 T3869 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[53]} 0 8
[junit4:junit4] 2> 1730839 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[54 (1438340753601331200)]} 0 1
[junit4:junit4] 2> 1730848 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[54 (1438340753608671232)]} 0 0
[junit4:junit4] 2> 1730850 T3822 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:26175/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[54 (1438340753608671232)]} 0 5
[junit4:junit4] 2> 1730851 T3855 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[54]} 0 9
[junit4:junit4] 2> 1730854 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[55 (1438340753617059840)]} 0 1
[junit4:junit4] 2> 1730861 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[55 (1438340753621254144)]} 0 1
[junit4:junit4] 2> 1730861 T3826 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[55 (1438340753621254144)]} 0 4
[junit4:junit4] 2> 1730864 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[56 (1438340753628594176)]} 0 0
[junit4:junit4] 2> 1730876 T3872 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:35206/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[56 (1438340753634885632)]} 0 1
[junit4:junit4] 2> 1730877 T3842 C1425 P35206 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[56 (1438340753634885632)]} 0 8
[junit4:junit4] 2> 1730878 T3826 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[56]} 0 11
[junit4:junit4] 2> 1730882 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[57 (1438340753646419968)]} 0 1
[junit4:junit4] 2> 1730892 T3872 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:35206/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[57 (1438340753653760000)]} 0 1
[junit4:junit4] 2> 1730893 T3836 C1425 P35206 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:26175/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[57 (1438340753653760000)]} 0 5
[junit4:junit4] 2> 1730894 T3855 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[57]} 0 9
[junit4:junit4] 2> 1730898 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[58 (1438340753663197184)]} 0 1
[junit4:junit4] 2> 1730907 T3872 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:35206/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[58 (1438340753670537216)]} 0 0
[junit4:junit4] 2> 1730908 T3836 C1425 P35206 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:26175/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[58 (1438340753670537216)]} 0 4
[junit4:junit4] 2> 1730909 T3855 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[58]} 0 8
[junit4:junit4] 2> 1730913 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[59 (1438340753678925824)]} 0 1
[junit4:junit4] 2> 1730923 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:22961/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[59 (1438340753686265856)]} 0 1
[junit4:junit4] 2> 1730924 T3820 C1423 P22961 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:35206/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[59 (1438340753686265856)]} 0 5
[junit4:junit4] 2> 1730925 T3838 C1425 P35206 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[59]} 0 9
[junit4:junit4] 2> 1730929 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[60 (1438340753695703040)]} 0 1
[junit4:junit4] 2> 1730938 T3872 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:35206/o_bh/e/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[60 (1438340753701994496)]} 0 1
[junit4:junit4] 2> 1730939 T3840 C1425 P35206 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={distrib.from=http://127.0.0.1:45459/o_bh/e/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[60 (1438340753701994496)]} 0 5
[junit4:junit4] 2> 1730940 T3869 C1424 P45459 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[60]} 0 8
[junit4:junit4] 2> 1730944 T3800 C1426 P25871 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_bh/e path=/update params={wt=javabin&version=2} {add=[61 (1438340753711431680)]} 0 1
[junit4:junit4] 2> 1730951 T3854 C1422 P26175 oasup.LogUpdateProcessor.finish [col
[...truncated too long message...]
2> 1940617 T3788 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 1940618 T3788 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 1940619 T3788 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 1940621 T3788 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 1940622 T3788 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 1940623 T3788 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty4 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty4;done=false>>]
[junit4:junit4] 2> 1940623 T3788 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty4
[junit4:junit4] 2> 1940624 T3788 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty4/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty4/index;done=false>>]
[junit4:junit4] 2> 1940624 T3788 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511/jetty4/index
[junit4:junit4] 2> 1940625 T3967 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89896294909280268-127.0.0.1:45459_o_bh%2Fe-n_0000000004) am no longer a leader.
[junit4:junit4] 2> 1940626 T3879 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1940626 T3879 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 1940627 T3788 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/o_bh/e,null}
[junit4:junit4] 2> 1940627 T3879 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 1940699 T3788 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4] 2> 1940701 T3788 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:34193 34193
[junit4:junit4] 2> 1940856 T3862 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 1940926 T3862 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1940927 T3862 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 1940928 T3788 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 25871
[junit4:junit4] 2> !!!! WARNING: best effort to remove /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371708601511 FAILED !!!!!
[junit4:junit4] 2> 1940929 T3788 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4] 2> 1940929 T3788 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:34193 34193
[junit4:junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ChaosMonkeyShardSplitTest -Dtests.method=testDistribSearch -Dtests.seed=BCC412BA92B002D4 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=cs_CZ -Dtests.timezone=America/Grand_Turk -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] ERROR 244s J1 | ChaosMonkeyShardSplitTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:22961/o_bh/e returned non ok status:500, message:Server Error
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([BCC412BA92B002D4:3D229CA2E5EF62E8]:0)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
[junit4:junit4] > at org.apache.solr.cloud.ChaosMonkeyShardSplitTest.doTest(ChaosMonkeyShardSplitTest.java:136)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> 1940949 T3788 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> NOTE: test params are: codec=Lucene40, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=cs_CZ, timezone=America/Grand_Turk
[junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=2,free=316422288,total=535822336
[junit4:junit4] 2> NOTE: All tests run in this JVM: [SuggesterWFSTTest, HighlighterConfigTest, TestArbitraryIndexDir, TestStressReorder, TestBinaryField, TestLMJelinekMercerSimilarityFactory, TestFiltering, SpellPossibilityIteratorTest, TestPostingsSolrHighlighter, SuggesterTSTTest, SpellCheckComponentTest, ZkSolrClientTest, DefaultValueUpdateProcessorTest, DOMUtilTest, TermVectorComponentDistributedTest, TestStressLucene, ConvertedLegacyTest, TestCSVResponseWriter, TestQueryTypes, IndexBasedSpellCheckerTest, TestSolr4Spatial, TestHashPartitioner, JSONWriterTest, MultiTermTest, TestRTGBase, ZkNodePropsTest, TestSchemaVersionResource, IndexReaderFactoryTest, TestDFRSimilarityFactory, TestPropInject, TestDefaultSimilarityFactory, SolrIndexConfigTest, CopyFieldTest, CurrencyFieldOpenExchangeTest, TestFastOutputStream, CollectionsAPIDistributedZkTest, TestReload, XsltUpdateRequestHandlerTest, TestLuceneMatchVersion, TestBinaryResponseWriter, TestFieldTypeResource, NotRequiredUniqueKeyTest, TestAnalyzedSuggestions, IndexSchemaRuntimeFieldTest, SolrCoreCheckLockOnStartupTest, TestSystemIdResolver, SolrPluginUtilsTest, CurrencyFieldXmlFileTest, TestOmitPositions, TestFieldResource, TestDynamicFieldResource, TestJmxIntegration, SimplePostToolTest, URLClassifyProcessorTest, NumericFieldsTest, FileUtilsTest, DirectSolrSpellCheckerTest, TestMaxScoreQueryParser, SpellCheckCollatorTest, LeaderElectionIntegrationTest, RequestHandlersTest, ChaosMonkeySafeLeaderTest, DirectUpdateHandlerTest, UpdateParamsTest, TestQuerySenderNoQuery, TestCSVLoader, OutputWriterTest, TestSerializedLuceneMatchVersion, TimeZoneUtilsTest, CoreContainerCoreInitFailuresTest, TestFieldTypeCollectionResource, BasicDistributedZk2Test, SyncSliceTest, OpenCloseCoreStressTest, OverseerTest, BasicZkTest, TestZkChroot, ShardRoutingCustomTest, TestDistributedSearch, AutoCommitTest, TestDistributedGrouping, TestRecovery, DistributedSpellCheckComponentTest, ZkControllerTest, TestJoin, SolrCmdDistributorTest, QueryElevationComponentTest, TestFunctionQuery, BasicFunctionalityTest, TestBadConfig, DistributedQueryElevationComponentTest, TestCoreDiscovery, SignatureUpdateProcessorFactoryTest, TestExtendedDismaxParser, TestFoldingMultitermQuery, SuggesterTest, NoCacheHeaderTest, TestAtomicUpdateErrorCases, QueryEqualityTest, TestRemoteStreaming, DistanceFunctionTest, TestSurroundQueryParser, DisMaxRequestHandlerTest, TestQueryUtils, DocumentAnalysisRequestHandlerTest, XmlUpdateRequestHandlerTest, PathHierarchyTokenizerFactoryTest, TestIndexingPerformance, MoreLikeThisHandlerTest, FieldAnalysisRequestHandlerTest, FastVectorHighlighterTest, UpdateRequestProcessorFactoryTest, TestPartialUpdateDeduplication, PingRequestHandlerTest, BadComponentTest, TestMergePolicyConfig, TestSolrDeletionPolicy2, SampleTest, TestDocSet, MinimalSchemaTest, TestElisionMultitermQuery, TestConfig, TestFuzzyAnalyzedSuggestions, DirectSolrConnectionTest, TestPhraseSuggestions, TestCodecSupport, SynonymTokenizerTest, EchoParamsTest, TestPerFieldSimilarity, TestNumberUtils, TestLMDirichletSimilarityFactory, TestBM25SimilarityFactory, TestFastWriter, LegacyHTMLStripCharFilterTest, TestJmxMonitoredMap, ResourceLoaderTest, OpenExchangeRatesOrgProviderTest, PluginInfoTest, TestFastLRUCache, DateMathParserTest, TestSolrXMLSerializer, PrimUtilsTest, TestSuggestSpellingConverter, DateFieldTest, ClusterStateTest, TestSolrJ, TestUtils, TestDocumentBuilder, SystemInfoHandlerTest, SolrTestCaseJ4Test, ChaosMonkeyShardSplitTest]
[junit4:junit4] Completed on J1 in 243.59s, 1 test, 1 error <<< FAILURES!
[...truncated 10 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:386: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:366: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:181: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:437: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1243: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:887: There were test failures: 297 suites, 1252 tests, 1 error, 13 ignored (7 assumptions)
Total time: 52 minutes 10 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4077 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4077/
2 tests failed.
FAILED: org.apache.solr.cloud.ShardSplitTest.testDistribSearch
Error Message:
Server at http://127.0.0.1:59967 returned non ok status:500, message:Server Error
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:59967 returned non ok status:500, message:Server Error
at __randomizedtesting.SeedInfo.seed([7D7AECDC52EC3B7:863120D5B271A38B]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
at org.apache.solr.cloud.ShardSplitTest.doTest(ShardSplitTest.java:150)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
FAILED: org.apache.solr.cloud.SyncSliceTest.testDistribSearch
Error Message:
shard1 is not consistent. Got 305 from http://127.0.0.1:19850/h_ow/h/collection1lastClient and got 254 from http://127.0.0.1:40036/h_ow/h/collection1
Stack Trace:
java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:19850/h_ow/h/collection1lastClient and got 254 from http://127.0.0.1:40036/h_ow/h/collection1
at __randomizedtesting.SeedInfo.seed([7D7AECDC52EC3B7:863120D5B271A38B]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9639 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.SyncSliceTest
[junit4:junit4] 2> 1013062 T3368 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /h_ow/h
[junit4:junit4] 2> 1013067 T3368 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./solrtest-SyncSliceTest-1371675029442
[junit4:junit4] 2> 1013068 T3368 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 1013069 T3369 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 1013170 T3368 oasc.ZkTestServer.run start zk server on port:22555
[junit4:junit4] 2> 1013171 T3368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1013174 T3375 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@21a8e6b3 name:ZooKeeperConnection Watcher:127.0.0.1:22555 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1013175 T3368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1013175 T3368 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 1013192 T3368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1013194 T3377 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5c47379b name:ZooKeeperConnection Watcher:127.0.0.1:22555/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1013194 T3368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1013194 T3368 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 1013197 T3368 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 1013207 T3368 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 1013209 T3368 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 1013211 T3368 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 1013211 T3368 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 1013216 T3368 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 1013217 T3368 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 1013339 T3368 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 1013340 T3368 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 1013342 T3368 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 1013343 T3368 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 1013351 T3368 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 1013352 T3368 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 1013354 T3368 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 1013354 T3368 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 1013356 T3368 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 1013357 T3368 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 1013364 T3368 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 1013365 T3368 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 1013367 T3368 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 1013367 T3368 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 1013542 T3368 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1013546 T3368 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:32062
[junit4:junit4] 2> 1013547 T3368 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1013547 T3368 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1013548 T3368 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371675029744
[junit4:junit4] 2> 1013548 T3368 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371675029744/solr.xml
[junit4:junit4] 2> 1013548 T3368 oasc.CoreContainer.<init> New CoreContainer 1799060924
[junit4:junit4] 2> 1013549 T3368 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371675029744/'
[junit4:junit4] 2> 1013549 T3368 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371675029744/'
[junit4:junit4] 2> 1013642 T3368 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1013643 T3368 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1013643 T3368 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1013644 T3368 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1013644 T3368 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1013644 T3368 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1013645 T3368 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1013645 T3368 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1013646 T3368 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1013646 T3368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1013658 T3368 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1013659 T3368 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:22555/solr
[junit4:junit4] 2> 1013659 T3368 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1013660 T3368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1013662 T3388 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@76d4672f name:ZooKeeperConnection Watcher:127.0.0.1:22555 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1013663 T3368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1013675 T3368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1013684 T3368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1013686 T3390 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@71ad8133 name:ZooKeeperConnection Watcher:127.0.0.1:22555/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1013686 T3368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1013688 T3368 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 1013697 T3368 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 1013705 T3368 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 1013707 T3368 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32062_h_ow%2Fh
[junit4:junit4] 2> 1013712 T3368 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:32062_h_ow%2Fh
[junit4:junit4] 2> 1013715 T3368 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 1013731 T3368 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 1013733 T3368 oasc.Overseer.start Overseer (id=89894094729838595-127.0.0.1:32062_h_ow%2Fh-n_0000000000) starting
[junit4:junit4] 2> 1013744 T3368 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 1013753 T3392 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 1013754 T3368 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 1013756 T3368 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 1013764 T3368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1013766 T3391 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 1013769 T3393 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1013770 T3393 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1015270 T3391 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1015271 T3391 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:32062_h_ow%2Fh",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:32062/h_ow/h"}
[junit4:junit4] 2> 1015271 T3391 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 1015271 T3391 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1015298 T3390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1015773 T3393 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371675029744/collection1
[junit4:junit4] 2> 1015773 T3393 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 1015774 T3393 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1015774 T3393 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 1015775 T3393 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371675029744/collection1/'
[junit4:junit4] 2> 1015776 T3393 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1371675029744/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1015777 T3393 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1371675029744/collection1/lib/README' to classloader
[junit4:junit4] 2> 1015814 T3393 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1015860 T3393 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1015962 T3393 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1015967 T3393 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1016423 T3393 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 1016432 T3393 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1016434 T3393 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1016444 T3393 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1016448 T3393 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1016451 T3393 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1016451 T3393 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1016452 T3393 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1016452 T3393 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1016453 T3393 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1016453 T3393 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1016453 T3393 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 1016453 T3393 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371675029744/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371675029441/control/data/
[junit4:junit4] 2> 1016454 T3393 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@637c72aa
[junit4:junit4] 2> 1016454 T3393 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 1016455 T3393 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/control/data
[junit4:junit4] 2> 1016456 T3393 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371675029441/control/data/index/
[junit4:junit4] 2> 1016456 T3393 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371675029441/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1016457 T3393 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/control/data/index
[junit4:junit4] 2> 1016463 T3393 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@21caec8e; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1016463 T3393 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1016465 T3393 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1016466 T3393 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1016466 T3393 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1016467 T3393 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1016467 T3393 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1016467 T3393 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1016468 T3393 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1016468 T3393 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1016468 T3393 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1016479 T3393 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1016486 T3393 oass.SolrIndexSearcher.<init> Opening Searcher@a2741a2 main
[junit4:junit4] 2> 1016487 T3393 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1016487 T3393 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1016490 T3394 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@a2741a2 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1016490 T3393 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1016491 T3393 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:32062/h_ow/h collection:control_collection shard:shard1
[junit4:junit4] 2> 1016491 T3393 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 1016517 T3393 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1016519 T3393 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1016519 T3393 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1016519 T3393 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:32062/h_ow/h/collection1/
[junit4:junit4] 2> 1016519 T3393 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1016519 T3393 oasc.SyncStrategy.syncToMe http://127.0.0.1:32062/h_ow/h/collection1/ has no replicas
[junit4:junit4] 2> 1016520 T3393 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:32062/h_ow/h/collection1/
[junit4:junit4] 2> 1016520 T3393 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 1016801 T3391 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1016824 T3390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1016840 T3393 oasc.ZkController.register We are http://127.0.0.1:32062/h_ow/h/collection1/ and leader is http://127.0.0.1:32062/h_ow/h/collection1/
[junit4:junit4] 2> 1016840 T3393 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:32062/h_ow/h
[junit4:junit4] 2> 1016840 T3393 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1016840 T3393 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1016841 T3393 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1016842 T3393 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1016843 T3368 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1016844 T3368 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1016844 T3368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1016855 T3368 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 1016856 T3368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1016858 T3397 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@445f2cf0 name:ZooKeeperConnection Watcher:127.0.0.1:22555/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1016858 T3368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1016859 T3368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1016861 T3368 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 1017039 T3368 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1017042 T3368 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:19850
[junit4:junit4] 2> 1017043 T3368 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1017043 T3368 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1017044 T3368 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371675033235
[junit4:junit4] 2> 1017044 T3368 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371675033235/solr.xml
[junit4:junit4] 2> 1017045 T3368 oasc.CoreContainer.<init> New CoreContainer 1869147660
[junit4:junit4] 2> 1017045 T3368 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371675033235/'
[junit4:junit4] 2> 1017046 T3368 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371675033235/'
[junit4:junit4] 2> 1017135 T3368 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1017136 T3368 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1017137 T3368 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1017137 T3368 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1017137 T3368 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1017138 T3368 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1017138 T3368 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1017138 T3368 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1017139 T3368 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1017139 T3368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1017149 T3368 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1017150 T3368 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:22555/solr
[junit4:junit4] 2> 1017151 T3368 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1017152 T3368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1017154 T3408 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@73161438 name:ZooKeeperConnection Watcher:127.0.0.1:22555 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1017155 T3368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1017166 T3368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1017173 T3368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1017175 T3410 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5d66f3cc name:ZooKeeperConnection Watcher:127.0.0.1:22555/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1017175 T3368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1017184 T3368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1018188 T3368 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:19850_h_ow%2Fh
[junit4:junit4] 2> 1018190 T3368 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:19850_h_ow%2Fh
[junit4:junit4] 2> 1018193 T3390 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1018193 T3397 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1018193 T3410 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1018194 T3390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1018211 T3411 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1018212 T3411 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1018336 T3391 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1018337 T3391 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:32062_h_ow%2Fh_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:32062_h_ow%2Fh",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:32062/h_ow/h"}
[junit4:junit4] 2> 1018340 T3391 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:19850_h_ow%2Fh",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:19850/h_ow/h"}
[junit4:junit4] 2> 1018340 T3391 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4] 2> 1018341 T3391 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1018353 T3390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1018353 T3410 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1018353 T3397 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1019214 T3411 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371675033235/collection1
[junit4:junit4] 2> 1019214 T3411 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1019215 T3411 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1019215 T3411 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1019216 T3411 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371675033235/collection1/'
[junit4:junit4] 2> 1019217 T3411 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1371675033235/collection1/lib/README' to classloader
[junit4:junit4] 2> 1019218 T3411 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1371675033235/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1019255 T3411 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1019298 T3411 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1019400 T3411 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1019405 T3411 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1019852 T3411 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 1019861 T3411 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1019864 T3411 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1019873 T3411 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1019877 T3411 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1019880 T3411 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1019881 T3411 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1019882 T3411 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1019882 T3411 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1019883 T3411 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1019883 T3411 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1019883 T3411 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 1019883 T3411 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371675033235/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty1/
[junit4:junit4] 2> 1019884 T3411 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@637c72aa
[junit4:junit4] 2> 1019884 T3411 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 1019885 T3411 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty1
[junit4:junit4] 2> 1019886 T3411 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty1/index/
[junit4:junit4] 2> 1019886 T3411 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1019887 T3411 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty1/index
[junit4:junit4] 2> 1019892 T3411 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b813436; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1019892 T3411 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1019895 T3411 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1019895 T3411 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1019896 T3411 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1019896 T3411 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1019897 T3411 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1019897 T3411 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1019897 T3411 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1019898 T3411 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1019898 T3411 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1019909 T3411 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1019916 T3411 oass.SolrIndexSearcher.<init> Opening Searcher@6821f4b0 main
[junit4:junit4] 2> 1019917 T3411 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1019917 T3411 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1019920 T3412 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6821f4b0 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1019922 T3411 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1019922 T3411 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:19850/h_ow/h collection:collection1 shard:shard1
[junit4:junit4] 2> 1019923 T3411 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 1019938 T3411 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1019950 T3411 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1019950 T3411 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1019950 T3411 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:19850/h_ow/h/collection1/
[junit4:junit4] 2> 1019951 T3411 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1019951 T3411 oasc.SyncStrategy.syncToMe http://127.0.0.1:19850/h_ow/h/collection1/ has no replicas
[junit4:junit4] 2> 1019951 T3411 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:19850/h_ow/h/collection1/
[junit4:junit4] 2> 1019951 T3411 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 1021369 T3391 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1021388 T3390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1021388 T3410 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1021388 T3397 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1021400 T3411 oasc.ZkController.register We are http://127.0.0.1:19850/h_ow/h/collection1/ and leader is http://127.0.0.1:19850/h_ow/h/collection1/
[junit4:junit4] 2> 1021400 T3411 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:19850/h_ow/h
[junit4:junit4] 2> 1021400 T3411 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1021401 T3411 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1021401 T3411 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1021402 T3411 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1021403 T3368 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1021404 T3368 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1021404 T3368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1021587 T3368 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1021590 T3368 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:10710
[junit4:junit4] 2> 1021590 T3368 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1021591 T3368 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1021591 T3368 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371675037789
[junit4:junit4] 2> 1021591 T3368 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371675037789/solr.xml
[junit4:junit4] 2> 1021592 T3368 oasc.CoreContainer.<init> New CoreContainer 132354271
[junit4:junit4] 2> 1021592 T3368 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371675037789/'
[junit4:junit4] 2> 1021593 T3368 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371675037789/'
[junit4:junit4] 2> 1021680 T3368 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1021681 T3368 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1021681 T3368 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1021682 T3368 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1021682 T3368 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1021682 T3368 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1021683 T3368 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1021683 T3368 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1021683 T3368 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1021684 T3368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1021698 T3368 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1021699 T3368 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:22555/solr
[junit4:junit4] 2> 1021700 T3368 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1021700 T3368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1021703 T3424 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7c59fd46 name:ZooKeeperConnection Watcher:127.0.0.1:22555 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1021703 T3368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1021720 T3368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1021729 T3368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1021731 T3426 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e194700 name:ZooKeeperConnection Watcher:127.0.0.1:22555/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1021731 T3368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1021735 T3368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1022738 T3368 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:10710_h_ow%2Fh
[junit4:junit4] 2> 1022740 T3368 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:10710_h_ow%2Fh
[junit4:junit4] 2> 1022742 T3410 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1022743 T3397 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1022743 T3426 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1022743 T3390 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1022743 T3397 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1022744 T3390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1022744 T3410 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1022758 T3427 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1022759 T3427 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1022893 T3391 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1022894 T3391 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:19850_h_ow%2Fh_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:19850_h_ow%2Fh",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:19850/h_ow/h"}
[junit4:junit4] 2> 1022899 T3391 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:10710_h_ow%2Fh",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:10710/h_ow/h"}
[junit4:junit4] 2> 1022900 T3391 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 1022900 T3391 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1022915 T3397 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1022915 T3390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1022915 T3426 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1022915 T3410 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1023762 T3427 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371675037789/collection1
[junit4:junit4] 2> 1023762 T3427 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1023763 T3427 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1023763 T3427 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1023764 T3427 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371675037789/collection1/'
[junit4:junit4] 2> 1023765 T3427 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1371675037789/collection1/lib/README' to classloader
[junit4:junit4] 2> 1023766 T3427 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1371675037789/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1023803 T3427 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1023848 T3427 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1023950 T3427 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1023955 T3427 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1024424 T3427 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 1024433 T3427 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1024436 T3427 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1024446 T3427 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1024450 T3427 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1024453 T3427 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1024454 T3427 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1024455 T3427 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1024455 T3427 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1024456 T3427 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1024456 T3427 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1024456 T3427 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 1024457 T3427 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371675037789/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty2/
[junit4:junit4] 2> 1024457 T3427 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@637c72aa
[junit4:junit4] 2> 1024457 T3427 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 1024458 T3427 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty2
[junit4:junit4] 2> 1024459 T3427 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty2/index/
[junit4:junit4] 2> 1024460 T3427 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1024460 T3427 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty2/index
[junit4:junit4] 2> 1024470 T3427 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b8eb46d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1024471 T3427 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1024474 T3427 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1024474 T3427 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1024475 T3427 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1024475 T3427 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1024476 T3427 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1024476 T3427 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1024476 T3427 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1024477 T3427 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1024477 T3427 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1024488 T3427 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1024496 T3427 oass.SolrIndexSearcher.<init> Opening Searcher@25c7c875 main
[junit4:junit4] 2> 1024497 T3427 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1024497 T3427 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1024501 T3428 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@25c7c875 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1024503 T3427 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1024503 T3427 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:10710/h_ow/h collection:collection1 shard:shard1
[junit4:junit4] 2> 1024507 T3427 oasc.ZkController.register We are http://127.0.0.1:10710/h_ow/h/collection1/ and leader is http://127.0.0.1:19850/h_ow/h/collection1/
[junit4:junit4] 2> 1024508 T3427 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:10710/h_ow/h
[junit4:junit4] 2> 1024508 T3427 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 1024508 T3427 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C1006 name=collection1 org.apache.solr.core.SolrCore@6c137082 url=http://127.0.0.1:10710/h_ow/h/collection1 node=127.0.0.1:10710_h_ow%2Fh C1006_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:10710_h_ow%2Fh, base_url=http://127.0.0.1:10710/h_ow/h}
[junit4:junit4] 2> 1024509 T3429 C1006 P10710 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 1024509 T3429 C1006 P10710 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 1024509 T3427 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1024509 T3429 C1006 P10710 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 1024509 T3429 C1006 P10710 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1024510 T3368 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1024511 T3368 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1024511 T3368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1024521 T3429 C1006 P10710 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1024531 T3400 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:10710_h_ow%2Fh_collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 1024694 T3368 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1024697 T3368 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:40036
[junit4:junit4] 2> 1024698 T3368 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1024698 T3368 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1024699 T3368 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371675040897
[junit4:junit4] 2> 1024699 T3368 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371675040897/solr.xml
[junit4:junit4] 2> 1024700 T3368 oasc.CoreContainer.<init> New CoreContainer 1776414430
[junit4:junit4] 2> 1024700 T3368 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371675040897/'
[junit4:junit4] 2> 1024701 T3368 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371675040897/'
[junit4:junit4] 2> 1024789 T3368 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1024789 T3368 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1024790 T3368 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1024790 T3368 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1024790 T3368 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1024791 T3368 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1024791 T3368 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1024792 T3368 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1024792 T3368 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1024792 T3368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1024804 T3368 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1024805 T3368 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:22555/solr
[junit4:junit4] 2> 1024805 T3368 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1024806 T3368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1024810 T3441 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@37708dc5 name:ZooKeeperConnection Watcher:127.0.0.1:22555 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1024811 T3368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1024822 T3368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1024831 T3368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1024833 T3443 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@8176621 name:ZooKeeperConnection Watcher:127.0.0.1:22555/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1024833 T3368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1024843 T3368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1025846 T3368 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40036_h_ow%2Fh
[junit4:junit4] 2> 1025858 T3368 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:40036_h_ow%2Fh
[junit4:junit4] 2> 1025863 T3410 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1025863 T3397 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1025863 T3426 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1025863 T3390 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1025863 T3443 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1025864 T3390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1025864 T3410 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1025863 T3426 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1025863 T3397 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1025880 T3444 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1025880 T3444 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1025924 T3391 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1025924 T3391 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:10710_h_ow%2Fh_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:10710_h_ow%2Fh",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:10710/h_ow/h"}
[junit4:junit4] 2> 1025928 T3391 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:40036_h_ow%2Fh",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:40036/h_ow/h"}
[junit4:junit4] 2> 1025928 T3391 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 1025928 T3391 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1025943 T3397 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1025943 T3426 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1025943 T3443 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1025943 T3390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1025943 T3410 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1026534 T3400 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:10710_h_ow%2Fh_collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 1026534 T3400 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:10710_h_ow%252Fh_collection1&state=recovering&nodeName=127.0.0.1:10710_h_ow%252Fh&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2003
[junit4:junit4] 2> 1026885 T3444 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371675040897/collection1
[junit4:junit4] 2> 1026885 T3444 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1026886 T3444 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1026886 T3444 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1026888 T3444 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371675040897/collection1/'
[junit4:junit4] 2> 1026889 T3444 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1371675040897/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1026890 T3444 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1371675040897/collection1/lib/README' to classloader
[junit4:junit4] 2> 1026945 T3444 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1027006 T3444 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1027109 T3444 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1027116 T3444 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1027651 T3444 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 1027660 T3444 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1027663 T3444 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1027675 T3444 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1027679 T3444 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1027683 T3444 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1027685 T3444 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1027685 T3444 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1027685 T3444 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1027686 T3444 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1027686 T3444 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1027687 T3444 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 1027687 T3444 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371675040897/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty3/
[junit4:junit4] 2> 1027687 T3444 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@637c72aa
[junit4:junit4] 2> 1027688 T3444 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 1027689 T3444 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty3
[junit4:junit4] 2> 1027689 T3444 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty3/index/
[junit4:junit4] 2> 1027690 T3444 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1027691 T3444 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty3/index
[junit4:junit4] 2> 1027696 T3444 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@40d5bd18; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1027697 T3444 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1027699 T3444 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1027700 T3444 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1027700 T3444 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1027701 T3444 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1027701 T3444 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1027701 T3444 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1027702 T3444 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1027702 T3444 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1027703 T3444 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1027715 T3444 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1027724 T3444 oass.SolrIndexSearcher.<init> Opening Searcher@8968c17 main
[junit4:junit4] 2> 1027725 T3444 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1027725 T3444 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1027729 T3445 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@8968c17 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1027731 T3444 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1027731 T3444 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:40036/h_ow/h collection:collection1 shard:shard1
[junit4:junit4] 2> 1027734 T3444 oasc.ZkController.register We are http://127.0.0.1:40036/h_ow/h/collection1/ and leader is http://127.0.0.1:19850/h_ow/h/collection1/
[junit4:junit4] 2> 1027735 T3444 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:40036/h_ow/h
[junit4:junit4] 2> 1027735 T3444 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 1027735 T3444 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C1007 name=collection1 org.apache.solr.core.SolrCore@6786c6f2 url=http://127.0.0.1:40036/h_ow/h/collection1 node=127.0.0.1:40036_h_ow%2Fh C1007_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:40036_h_ow%2Fh, base_url=http://127.0.0.1:40036/h_ow/h}
[junit4:junit4] 2> 1027735 T3446 C1007 P40036 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 1027736 T3444 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1027736 T3446 C1007 P40036 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 1027737 T3446 C1007 P40036 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 1027737 T3446 C1007 P40036 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1027737 T3368 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1027738 T3368 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1027739 T3446 C1007 P40036 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1027739 T3368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1027760 T3401 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:40036_h_ow%2Fh_collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 1028030 T3368 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1028033 T3368 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:20404
[junit4:junit4] 2> 1028034 T3368 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1028034 T3368 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1028035 T3368 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371675044132
[junit4:junit4] 2> 1028035 T3368 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371675044132/solr.xml
[junit4:junit4] 2> 1028036 T3368 oasc.CoreContainer.<init> New CoreContainer 1668662114
[junit4:junit4] 2> 1028037 T3368 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371675044132/'
[junit4:junit4] 2> 1028037 T3368 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371675044132/'
[junit4:junit4] 2> 1028166 T3368 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1028167 T3368 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1028167 T3368 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1028168 T3368 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1028168 T3368 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1028169 T3368 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1028169 T3368 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1028170 T3368 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1028171 T3368 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1028171 T3368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1028188 T3368 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1028189 T3368 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:22555/solr
[junit4:junit4] 2> 1028189 T3368 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1028190 T3368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1028193 T3458 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e1d94aa name:ZooKeeperConnection Watcher:127.0.0.1:22555 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1028194 T3368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1028204 T3368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1028212 T3368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1028214 T3460 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@64467641 name:ZooKeeperConnection Watcher:127.0.0.1:22555/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1028214 T3368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1028225 T3368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C1008 name=collection1 org.apache.solr.core.SolrCore@6c137082 url=http://127.0.0.1:10710/h_ow/h/collection1 node=127.0.0.1:10710_h_ow%2Fh C1008_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:10710_h_ow%2Fh, base_url=http://127.0.0.1:10710/h_ow/h}
[junit4:junit4] 2> 1028536 T3429 C1008 P10710 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:19850/h_ow/h/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 1028536 T3429 C1008 P10710 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:10710/h_ow/h START replicas=[http://127.0.0.1:19850/h_ow/h/collection1/] nUpdates=100
[junit4:junit4] 2> 1028537 T3429 C1008 P10710 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 1028538 T3429 C1008 P10710 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 1028538 T3429 C1008 P10710 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 1028538 T3429 C1008 P10710 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 1028539 T3429 C1008 P10710 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 1028539 T3429 C1008 P10710 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:19850/h_ow/h/collection1/. core=collection1
[junit4:junit4] 2> 1028539 T3429 C1008 P10710 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C1009 name=collection1 org.apache.solr.core.SolrCore@38d6c725 url=http://127.0.0.1:19850/h_ow/h/collection1 node=127.0.0.1:19850_h_ow%2Fh C1009_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:19850_h_ow%2Fh, base_url=http://127.0.0.1:19850/h_ow/h, leader=true}
[junit4:junit4] 2> 1028544 T3402 C1009 P19850 oasc.SolrCore.execute [collection1] webapp=/h_ow/h path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 1028551 T3403 C1009 P19850 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1028555 T3403 C1009 P19850 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b813436; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1028556 T3403 C1009 P19850 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1028559 T3403 C1009 P19850 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b813436; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b813436; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1028560 T3403 C1009 P19850 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1028561 T3403 C1009 P19850 oass.SolrIndexSearcher.<init> Opening Searcher@61fa5654 realtime
[junit4:junit4] 2> 1028562 T3403 C1009 P19850 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1028562 T3403 C1009 P19850 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 11
[junit4:junit4] 2> 1028563 T3429 C1008 P10710 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1028563 T3429 C1008 P10710 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 1028565 T3404 C1009 P19850 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1028566 T3404 C1009 P19850 oasc.SolrCore.execute [collection1] webapp=/h_ow/h path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2
[junit4:junit4] 2> 1028566 T3429 C1008 P10710 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 1028566 T3429 C1008 P10710 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 1028567 T3429 C1008 P10710 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 1028571 T3404 C1009 P19850 oasc.SolrCore.execute [collection1] webapp=/h_ow/h path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1
[junit4:junit4] 2> 1028571 T3429 C1008 P10710 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 1028573 T3429 C1008 P10710 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty2/index.20130619155044946
[junit4:junit4] 2> 1028574 T3429 C1008 P10710 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty2/index.20130619155044946 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6a85b6a; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4] 2> 1028578 T3404 C1009 P19850 oasc.SolrCore.execute [collection1] webapp=/h_ow/h path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 1028580 T3429 C1008 P10710 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 1028582 T3429 C1008 P10710 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 1028582 T3429 C1008 P10710 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 1028587 T3429 C1008 P10710 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b8eb46d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b8eb46d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1028587 T3429 C1008 P10710 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1028588 T3429 C1008 P10710 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 1028589 T3429 C1008 P10710 oass.SolrIndexSearcher.<init> Opening Searcher@24e7ab83 main
[junit4:junit4] 2> 1028590 T3428 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@24e7ab83 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1028591 T3429 C1008 P10710 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty2/index.20130619155044946 [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty2/index.20130619155044946;done=true>>]
[junit4:junit4] 2> 1028591 T3429 C1008 P10710 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty2/index.20130619155044946
[junit4:junit4] 2> 1028591 T3429 C1008 P10710 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty2/index.20130619155044946
[junit4:junit4] 2> 1028592 T3429 C1008 P10710 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 1028592 T3429 C1008 P10710 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 1028593 T3429 C1008 P10710 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1028593 T3429 C1008 P10710 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1028594 T3429 C1008 P10710 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 1028952 T3391 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1028953 T3391 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:40036_h_ow%2Fh_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:40036_h_ow%2Fh",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:40036/h_ow/h"}
[junit4:junit4] 2> 1028977 T3391 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:10710_h_ow%2Fh_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:10710_h_ow%2Fh",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:10710/h_ow/h"}
[junit4:junit4] 2> 1029046 T3397 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1029046 T3460 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1029047 T3443 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1029046 T3426 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1029046 T3410 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1029046 T3390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1029230 T3368 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:20404_h_ow%2Fh
[junit4:junit4] 2> 1029232 T3368 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:20404_h_ow%2Fh
[junit4:junit4] 2> 1029235 T3443 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1029235 T3397 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1029235 T3410 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1029236 T3397 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1029235 T3426 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1029235 T3390 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1029235 T3460 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1029237 T3390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1029236 T3426 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1029237 T3460 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1029258 T3443 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1029259 T3410 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1029268 T3463 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1029268 T3463 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1029763 T3401 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:40036_h_ow%2Fh_collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 1029763 T3401 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:40036_h_ow%252Fh_collection1&state=recovering&nodeName=127.0.0.1:40036_h_ow%252Fh&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2003
[junit4:junit4] 2> 1030643 T3391 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1030644 T3391 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:20404_h_ow%2Fh",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:20404/h_ow/h"}
[junit4:junit4] 2> 1030644 T3391 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 1030645 T3391 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1030665 T3397 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1030665 T3443 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1030665 T3426 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1030665 T3390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1030665 T3410 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1030665 T3460 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1031274 T3463 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371675044132/collection1
[junit4:junit4] 2> 1031274 T3463 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1031275 T3463 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1031275 T3463 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1031277 T3463 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371675044132/collection1/'
[junit4:junit4] 2> 1031278 T3463 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1371675044132/collection1/lib/README' to classloader
[junit4:junit4] 2> 1031279 T3463 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1371675044132/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1031367 T3463 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1031430 T3463 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1031532 T3463 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1031539 T3463 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> ASYNC NEW_CORE C1010 name=collection1 org.apache.solr.core.SolrCore@6786c6f2 url=http://127.0.0.1:40036/h_ow/h/collection1 node=127.0.0.1:40036_h_ow%2Fh C1010_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:40036_h_ow%2Fh, base_url=http://127.0.0.1:40036/h_ow/h}
[junit4:junit4] 2> 1031765 T3446 C1010 P40036 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:19850/h_ow/h/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 1031765 T3446 C1010 P40036 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:40036/h_ow/h START replicas=[http://127.0.0.1:19850/h_ow/h/collection1/] nUpdates=100
[junit4:junit4] 2> 1031766 T3446 C1010 P40036 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 1031766 T3446 C1010 P40036 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 1031766 T3446 C1010 P40036 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 1031767 T3446 C1010 P40036 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 1031767 T3446 C1010 P40036 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 1031767 T3446 C1010 P40036 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:19850/h_ow/h/collection1/. core=collection1
[junit4:junit4] 2> 1031767 T3446 C1010 P40036 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C1011 name=collection1 org.apache.solr.core.SolrCore@38d6c725 url=http://127.0.0.1:19850/h_ow/h/collection1 node=127.0.0.1:19850_h_ow%2Fh C1011_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:19850_h_ow%2Fh, base_url=http://127.0.0.1:19850/h_ow/h, leader=true}
[junit4:junit4] 2> 1031769 T3402 C1011 P19850 oasc.SolrCore.execute [collection1] webapp=/h_ow/h path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 1031785 T3405 C1011 P19850 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1031792 T3405 C1011 P19850 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b813436; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b813436; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1031793 T3405 C1011 P19850 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1031794 T3405 C1011 P19850 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1031795 T3405 C1011 P19850 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4] 2> 1031796 T3446 C1010 P40036 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1031796 T3446 C1010 P40036 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 1031798 T3404 C1011 P19850 oasc.SolrCore.execute [collection1] webapp=/h_ow/h path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
[junit4:junit4] 2> 1031799 T3446 C1010 P40036 oash.SnapPuller.fetchLatestIndex Master's generation: 3
[junit4:junit4] 2> 1031799 T3446 C1010 P40036 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 1031799 T3446 C1010 P40036 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 1031803 T3404 C1011 P19850 oasc.SolrCore.execute [collection1] webapp=/h_ow/h path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=3&version=2} status=0 QTime=1
[junit4:junit4] 2> 1031803 T3446 C1010 P40036 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 1031805 T3446 C1010 P40036 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty3/index.20130619155048178
[junit4:junit4] 2> 1031806 T3446 C1010 P40036 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty3/index.20130619155048178 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2746e7c5; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4] 2> 1031811 T3404 C1011 P19850 oasc.SolrCore.execute [collection1] webapp=/h_ow/h path=/replication params={file=segments_3&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=3} status=0 QTime=0
[junit4:junit4] 2> 1031813 T3446 C1010 P40036 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 1031816 T3446 C1010 P40036 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 1031816 T3446 C1010 P40036 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 1031822 T3446 C1010 P40036 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@40d5bd18; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@40d5bd18; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1031822 T3446 C1010 P40036 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1031823 T3446 C1010 P40036 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 1031824 T3446 C1010 P40036 oass.SolrIndexSearcher.<init> Opening Searcher@56cbf156 main
[junit4:junit4] 2> 1031825 T3445 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@56cbf156 main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 1031826 T3446 C1010 P40036 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty3/index.20130619155048178 [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty3/index.20130619155048178;done=true>>]
[junit4:junit4] 2> 1031826 T3446 C1010 P40036 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty3/index.20130619155048178
[junit4:junit4] 2> 1031827 T3446 C1010 P40036 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty3/index.20130619155048178
[junit4:junit4] 2> 1031828 T3446 C1010 P40036 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 1031828 T3446 C1010 P40036 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 1031829 T3446 C1010 P40036 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1031829 T3446 C1010 P40036 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1031830 T3446 C1010 P40036 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 1032135 T3463 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 1032144 T3463 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1032147 T3463 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1032159 T3463 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1032170 T3391 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1032171 T3391 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:40036_h_ow%2Fh_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:40036_h_ow%2Fh",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:40036/h_ow/h"}
[junit4:junit4] 2> 1032173 T3463 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1032177 T3463 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1032185 T3390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1032185 T3426 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1032185 T3460 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1032185 T3443 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1032185 T3410 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1032185 T3397 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1032185 T3463 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1032187 T3463 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1032187 T3463 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1032188 T3463 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1032188 T3463 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1032189 T3463 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 1032189 T3463 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371675044132/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty4/
[junit4:junit4] 2> 1032189 T3463 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@637c72aa
[junit4:junit4] 2> 1032190 T3463 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 1032191 T3463 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty4
[junit4:junit4] 2> 1032193 T3463 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty4/index/
[junit4:junit4] 2> 1032193 T3463 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1032194 T3463 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty4/index
[junit4:junit4] 2> 1032200 T3463 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@750fbff9; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1032200 T3463 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1032203 T3463 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1032203 T3463 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1032204 T3463 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1032205 T3463 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1032205 T3463 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1032205 T3463 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1032208 T3463 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1032208 T3463 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1032208 T3463 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1032219 T3463 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1032226 T3463 oass.SolrIndexSearcher.<init> Opening Searcher@7aa2d718 main
[junit4:junit4] 2> 1032227 T3463 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1032227 T3463 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1032230 T3466 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7aa2d718 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1032232 T3463 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1032232 T3463 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:20404/h_ow/h collection:collection1 shard:shard1
[junit4:junit4] 2> 1032236 T3463 oasc.ZkController.register We are http://127.0.0.1:20404/h_ow/h/collection1/ and leader is http://127.0.0.1:19850/h_ow/h/collection1/
[junit4:junit4] 2> 1032236 T3463 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:20404/h_ow/h
[junit4:junit4] 2> 1032236 T3463 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 1032236 T3463 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C1012 name=collection1 org.apache.solr.core.SolrCore@13fa7110 url=http://127.0.0.1:20404/h_ow/h/collection1 node=127.0.0.1:20404_h_ow%2Fh C1012_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:20404_h_ow%2Fh, base_url=http://127.0.0.1:20404/h_ow/h}
[junit4:junit4] 2> 1032236 T3467 C1012 P20404 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 1032237 T3463 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1032237 T3467 C1012 P20404 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 1032237 T3467 C1012 P20404 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 1032237 T3467 C1012 P20404 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1032238 T3368 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1032239 T3368 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1032238 T3467 C1012 P20404 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1032239 T3368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1032255 T3368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1032256 T3406 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:20404_h_ow%2Fh_collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 1032257 T3368 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 1032257 T3368 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 1032258 T3368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1033260 T3368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1033690 T3391 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1033691 T3391 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:20404_h_ow%2Fh_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:20404_h_ow%2Fh",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:20404/h_ow/h"}
[junit4:junit4] 2> 1033705 T3460 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1033705 T3443 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1033705 T3390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1033706 T3397 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1033706 T3426 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1033706 T3410 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1034261 T3406 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:20404_h_ow%2Fh_collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 1034261 T3406 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:20404_h_ow%252Fh_collection1&state=recovering&nodeName=127.0.0.1:20404_h_ow%252Fh&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2005
[junit4:junit4] 2> 1034263 T3368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1035266 T3368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C1012_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:20404_h_ow%2Fh, base_url=http://127.0.0.1:20404/h_ow/h}
[junit4:junit4] 2> 1036263 T3467 C1012 P20404 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:19850/h_ow/h/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 1036263 T3467 C1012 P20404 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:20404/h_ow/h START replicas=[http://127.0.0.1:19850/h_ow/h/collection1/] nUpdates=100
[junit4:junit4] 2> 1036264 T3467 C1012 P20404 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 1036264 T3467 C1012 P20404 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 1036264 T3467 C1012 P20404 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 1036265 T3467 C1012 P20404 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 1036265 T3467 C1012 P20404 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 1036265 T3467 C1012 P20404 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:19850/h_ow/h/collection1/. core=collection1
[junit4:junit4] 2> 1036265 T3467 C1012 P20404 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1036266 T3402 C1011 P19850 oasc.SolrCore.execute [collection1] webapp=/h_ow/h path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 1036269 T3368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1036276 T3400 C1011 P19850 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1036296 T3400 C1011 P19850 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b813436; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b813436; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b813436; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 1036297 T3400 C1011 P19850 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 1036298 T3400 C1011 P19850 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1036298 T3400 C1011 P19850 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 22
[junit4:junit4] 2> 1036299 T3467 C1012 P20404 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1036300 T3467 C1012 P20404 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 1036302 T3404 C1011 P19850 oasc.SolrCore.execute [collection1] webapp=/h_ow/h path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
[junit4:junit4] 2> 1036310 T3467 C1012 P20404 oash.SnapPuller.fetchLatestIndex Master's generation: 4
[junit4:junit4] 2> 1036311 T3467 C1012 P20404 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 1036311 T3467 C1012 P20404 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 1036314 T3404 C1011 P19850 oasc.SolrCore.execute [collection1] webapp=/h_ow/h path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=4&version=2} status=0 QTime=1
[junit4:junit4] 2> 1036315 T3467 C1012 P20404 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 1036317 T3467 C1012 P20404 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty4/index.20130619155052690
[junit4:junit4] 2> 1036318 T3467 C1012 P20404 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty4/index.20130619155052690 lockFactory=org.apache.lucene.store.NativeFSLockFactory@385b5dec; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4] 2> 1036322 T3404 C1011 P19850 oasc.SolrCore.execute [collection1] webapp=/h_ow/h path=/replication params={file=segments_4&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=4} status=0 QTime=0
[junit4:junit4] 2> 1036324 T3467 C1012 P20404 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 1036327 T3467 C1012 P20404 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 1036327 T3467 C1012 P20404 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 1036332 T3467 C1012 P20404 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@750fbff9; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@750fbff9; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 1036332 T3467 C1012 P20404 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 1036333 T3467 C1012 P20404 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 1036334 T3467 C1012 P20404 oass.SolrIndexSearcher.<init> Opening Searcher@11995d5 main
[junit4:junit4] 2> 1036335 T3466 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@11995d5 main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 1036336 T3467 C1012 P20404 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty4/index.20130619155052690 [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty4/index.20130619155052690;done=true>>]
[junit4:junit4] 2> 1036336 T3467 C1012 P20404 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty4/index.20130619155052690
[junit4:junit4] 2> 1036336 T3467 C1012 P20404 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty4/index.20130619155052690
[junit4:junit4] 2> 1036338 T3467 C1012 P20404 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 1036338 T3467 C1012 P20404 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 1036338 T3467 C1012 P20404 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1036338 T3467 C1012 P20404 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1036340 T3467 C1012 P20404 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 1036712 T3391 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1036713 T3391 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:20404_h_ow%2Fh_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:20404_h_ow%2Fh",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:20404/h_ow/h"}
[junit4:junit4] 2> 1036728 T3443 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1036728 T3460 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1036728 T3390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1036728 T3397 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1036728 T3426 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1036728 T3410 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1037272 T3368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1037274 T3368 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C1013 name=collection1 org.apache.solr.core.SolrCore@489dfbef url=http://127.0.0.1:32062/h_ow/h/collection1 node=127.0.0.1:32062_h_ow%2Fh C1013_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:32062_h_ow%2Fh, base_url=http://127.0.0.1:32062/h_ow/h, leader=true}
[junit4:junit4] 2> 1037284 T3380 C1013 P32062 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1037289 T3380 C1013 P32062 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@21caec8e; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1037289 T3380 C1013 P32062 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1037294 T3380 C1013 P32062 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@21caec8e; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@21caec8e; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1037294 T3380 C1013 P32062 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1037296 T3380 C1013 P32062 oass.SolrIndexSearcher.<init> Opening Searcher@74d01842 main
[junit4:junit4] 2> 1037296 T3380 C1013 P32062 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1037297 T3394 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@74d01842 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1037298 T3380 C1013 P32062 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 15
[junit4:junit4] 2> ASYNC NEW_CORE C1014 name=collection1 org.apache.solr.core.SolrCore@6c137082 url=http://127.0.0.1:10710/h_ow/h/collection1 node=127.0.0.1:10710_h_ow%2Fh C1014_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:10710_h_ow%2Fh, base_url=http://127.0.0.1:10710/h_ow/h}
[junit4:junit4] 2> 1037319 T3416 C1014 P10710 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:19850/h_ow/h/collection1/, StdNode: http://127.0.0.1:10710/h_ow/h/collection1/, StdNode: http://127.0.0.1:40036/h_ow/h/collection1/, StdNode: http://127.0.0.1:20404/h_ow/h/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 1037323 T3417 C1014 P10710 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1037323 T3403 C1011 P19850 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> C1010_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:40036_h_ow%2Fh, base_url=http://127.0.0.1:40036/h_ow/h}
[junit4:junit4] 2> 1037337 T3433 C1010 P40036 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1037340 T3417 C1014 P10710 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b8eb46d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b8eb46d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1037340 T3417 C1014 P10710 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1037341 T3403 C1011 P19850 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=4
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b813436; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b813436; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b813436; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b813436; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> 1037342 T3403 C1011 P19850 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 1037343 T3433 C1010 P40036 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@40d5bd18; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@40d5bd18; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 1037343 T3417 C1014 P10710 oass.SolrIndexSearcher.<init> Opening Searcher@77c0b1d6 main
[junit4:junit4] 2> 1037344 T3403 C1011 P19850 oass.SolrIndexSearcher.<init> Opening Searcher@7d5669fe main
[junit4:junit4] 2> 1037344 T3433 C1010 P40036 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 1037344 T3403 C1011 P19850 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> C1012_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:20404_h_ow%2Fh, base_url=http://127.0.0.1:20404/h_ow/h}
[junit4:junit4] 2> 1037344 T3450 C1012 P20404 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1037344 T3417 C1014 P10710 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1037346 T3433 C1010 P40036 oass.SolrIndexSearcher.<init> Opening Searcher@3b8fd087 main
[junit4:junit4] 2> 1037345 T3412 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7d5669fe main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1037345 T3428 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@77c0b1d6 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1037347 T3433 C1010 P40036 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1037348 T3417 C1014 P10710 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 25
[junit4:junit4] 2> 1037348 T3445 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3b8fd087 main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 1037348 T3403 C1011 P19850 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 25
[junit4:junit4] 2> 1037349 T3433 C1010 P40036 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 13
[junit4:junit4] 2> 1037353 T3450 C1012 P20404 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@750fbff9; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@750fbff9; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> 1037353 T3450 C1012 P20404 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 1037355 T3450 C1012 P20404 oass.SolrIndexSearcher.<init> Opening Searcher@7a84cade main
[junit4:junit4] 2> 1037356 T3450 C1012 P20404 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1037357 T3466 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7a84cade main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 1037357 T3450 C1012 P20404 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 13
[junit4:junit4] 2> 1037358 T3416 C1014 P10710 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 39
[junit4:junit4] 2> 1037359 T3368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1037362 T3401 C1011 P19850 oasc.SolrCore.execute [collection1] webapp=/h_ow/h path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 1037364 T3418 C1014 P10710 oasc.SolrCore.execute [collection1] webapp=/h_ow/h path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 1037366 T3434 C1010 P40036 oasc.SolrCore.execute [collection1] webapp=/h_ow/h path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 1037368 T3451 C1012 P20404 oasc.SolrCore.execute [collection1] webapp=/h_ow/h path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 1039374 T3380 C1013 P32062 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1438305543254966272)} 0 2
[junit4:junit4] 2> 1039381 T3452 C1012 P20404 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={update.distrib=FROMLEADER&_version_=-1438305543259160576&update.from=http://127.0.0.1:19850/h_ow/h/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438305543259160576)} 0 1
[junit4:junit4] 2> 1039382 T3419 C1014 P10710 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={update.distrib=FROMLEADER&_version_=-1438305543259160576&update.from=http://127.0.0.1:19850/h_ow/h/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438305543259160576)} 0 2
[junit4:junit4] 2> 1039383 T3435 C1010 P40036 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={update.distrib=FROMLEADER&_version_=-1438305543259160576&update.from=http://127.0.0.1:19850/h_ow/h/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438305543259160576)} 0 2
[junit4:junit4] 2> 1039384 T3405 C1011 P19850 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1438305543259160576)} 0 8
[junit4:junit4] 2> 1039389 T3380 C1013 P32062 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={wt=javabin&version=2} {add=[0 (1438305543269646336)]} 0 2
[junit4:junit4] 2> 1039399 T3419 C1014 P10710 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={distrib.from=http://127.0.0.1:19850/h_ow/h/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438305543275937792)]} 0 2
[junit4:junit4] 2> 1039399 T3452 C1012 P20404 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={distrib.from=http://127.0.0.1:19850/h_ow/h/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438305543275937792)]} 0 2
[junit4:junit4] 2> 1039400 T3435 C1010 P40036 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={distrib.from=http://127.0.0.1:19850/h_ow/h/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438305543275937792)]} 0 3
[junit4:junit4] 2> 1039401 T3406 C1011 P19850 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={distrib.from=http://127.0.0.1:20404/h_ow/h/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[0 (1438305543275937792)]} 0 9
[junit4:junit4] 2> 1039401 T3453 C1012 P20404 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={wt=javabin&version=2} {add=[0]} 0 10
[junit4:junit4] 2> 1039406 T3380 C1013 P32062 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={wt=javabin&version=2} {add=[1 (1438305543288520704)]} 0 1
[junit4:junit4] 2> 1039413 T3419 C1014 P10710 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={distrib.from=http://127.0.0.1:19850/h_ow/h/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438305543292715008)]} 0 1
[junit4:junit4] 2> 1039414 T3452 C1012 P20404 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={distrib.from=http://127.0.0.1:19850/h_ow/h/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438305543292715008)]} 0 1
[junit4:junit4] 2> 1039414 T3435 C1010 P40036 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={distrib.from=http://127.0.0.1:19850/h_ow/h/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438305543292715008)]} 0 1
[junit4:junit4] 2> 1039415 T3400 C1011 P19850 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={distrib.from=http://127.0.0.1:40036/h_ow/h/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1438305543292715008)]} 0 6
[junit4:junit4] 2> 1039416 T3436 C1010 P40036 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={wt=javabin&version=2} {add=[1]} 0 9
[junit4:junit4] 2> 1039419 T3380 C1013 P32062 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={wt=javabin&version=2} {add=[2 (1438305543302152192)]} 0 1
[junit4:junit4] 2> 1039423 T3403 C1011 P19850 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:10710/h_ow/h/collection1/]
[junit4:junit4] 2> 1039424 T3403 C1011 P19850 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:10710/h_ow/h/collection1/ against:[http://127.0.0.1:10710/h_ow/h/collection1/] result:true
[junit4:junit4] 2> 1039424 T3403 C1011 P19850 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:40036/h_ow/h/collection1/ against:[http://127.0.0.1:10710/h_ow/h/collection1/] result:false
[junit4:junit4] 2> 1039424 T3403 C1011 P19850 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:20404/h_ow/h/collection1/ against:[http://127.0.0.1:10710/h_ow/h/collection1/] result:false
[junit4:junit4] 2> 1039427 T3452 C1012 P20404 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={distrib.from=http://127.0.0.1:19850/h_ow/h/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:10710/h_ow/h/collection1/&wt=javabin&version=2} {add=[2 (1438305543308443648)]} 0 0
[junit4:junit4] 2> 1039428 T3435 C1010 P40036 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={distrib.from=http://127.0.0.1:19850/h_ow/h/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:10710/h_ow/h/collection1/&wt=javabin&version=2} {add=[2 (1438305543308443648)]} 0 1
[junit4:junit4] 2> 1039429 T3403 C1011 P19850 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={distrib.from=http://127.0.0.1:10710/h_ow/h/collection1/&update.distrib=TOLEADER&test.distrib.skip.servers=http://127.0.0.1:10710/h_ow/h/collection1/&wt=javabin&version=2} {add=[2 (1438305543308443648)]} 0 6
[junit4:junit4] 2> 1039430 T3416 C1014 P10710 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={test.distrib.skip.servers=http://127.0.0.1:10710/h_ow/h/collection1/&wt=javabin&version=2} {add=[2]} 0 9
[junit4:junit4] 2> 1039433 T3380 C1013 P32062 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={wt=javabin&version=2} {add=[3 (1438305543316832256)]} 0 1
[junit4:junit4] 2> 1039437 T3406 C1011 P19850 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:10710/h_ow/h/collection1/]
[junit4:junit4] 2> 1039438 T3406 C1011 P19850 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:10710/h_ow/h/collection1/ against:[http://127.0.0.1:10710/h_ow/h/collection1/] result:true
[junit4:junit4] 2> 1039439 T3406 C1011 P19850 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:40036/h_ow/h/collection1/ against:[http://127.0.0.1:10710/h_ow/h/collection1/] result:false
[junit4:junit4] 2> 1039439 T3406 C1011 P19850 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:20404/h_ow/h/collection1/ against:[http://127.0.0.1:10710/h_ow/h/collection1/] result:false
[junit4:junit4] 2> 1039443 T3452 C1012 P20404 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={distrib.from=http://127.0.0.1:19850/h_ow/h/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:10710/h_ow/h/collection1/&wt=javabin&version=2} {add=[3 (1438305543324172288)]} 0 1
[junit4:junit4] 2> 1039443 T3435 C1010 P40036 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={distrib.from=http://127.0.0.1:19850/h_ow/h/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:10710/h_ow/h/collection1/&wt=javabin&version=2} {add=[3 (1438305543324172288)]} 0 1
[junit4:junit4] 2> 1039444 T3406 C1011 P19850 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={distrib.from=http://127.0.0.1:20404/h_ow/h/collection1/&update.distrib=TOLEADER&test.distrib.skip.servers=http://127.0.0.1:10710/h_ow/h/collection1/&wt=javabin&version=2} {add=[3 (1438305543324172288)]} 0 7
[junit4:junit4] 2> 1039445 T3453 C1012 P20404 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={test.distrib.skip.servers=http://127.0.0.1:10710/h_ow/h/collection1/&test.distrib.skip.servers=http://127.0.0.1:40036/h_ow/h/collection1/&wt=javabin&version=2} {add=[3]} 0 10
[junit4:junit4] 2> 1039448 T3380 C1013 P32062 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1039493 T3380 C1013 P32062 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@21caec8e; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@21caec8e; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1039494 T3380 C1013 P32062 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1039498 T3380 C1013 P32062 oass.SolrIndexSearcher.<init> Opening Searcher@7b32cb62 main
[junit4:junit4] 2> 1039499 T3380 C1013 P32062 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1039499 T3394 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7b32cb62 main{StandardDirectoryReader(segments_3:3:nrt _0(5.0):c4)}
[junit4:junit4] 2> 1039500 T3380 C1013 P32062 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 52
[junit4:junit4] 2> 1039503 T3436 C1010 P40036 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:19850/h_ow/h/collection1/, StdNode: http://127.0.0.1:10710/h_ow/h/collection1/, StdNode: http://127.0.0.1:40036/h_ow/h/collection1/, StdNode: http://127.0.0.1:20404/h_ow/h/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 1039506 T3420 C1014 P10710 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1039506 T3454 C1012 P20404 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1039507 T3400 C1011 P19850 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1039506 T3437 C1010 P40036 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1039578 T3454 C1012 P20404 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@750fbff9; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@750fbff9; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_6,generation=6}
[junit4:junit4] 2> 1039578 T3420 C1014 P10710 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b8eb46d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b8eb46d; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 1039579 T3454 C1012 P20404 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 1039580 T3420 C1014 P10710 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 1039585 T3454 C1012 P20404 oass.SolrIndexSearcher.<init> Opening Searcher@5459b2b main
[junit4:junit4] 2> 1039585 T3420 C1014 P10710 oass.SolrIndexSearcher.<init> Opening Searcher@4bfa3075 main
[junit4:junit4] 2> 1039585 T3454 C1012 P20404 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1039586 T3400 C1011 P19850 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=5
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b813436; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b813436; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b813436; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b813436; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b813436; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_6,generation=6}
[junit4:junit4] 2> 1039586 T3420 C1014 P10710 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1039587 T3400 C1011 P19850 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 1039586 T3428 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4bfa3075 main{StandardDirectoryReader(segments_4:3:nrt _0(5.0):c2)}
[junit4:junit4] 2> 1039586 T3437 C1010 P40036 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@40d5bd18; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371675029441/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@40d5bd18; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> 1039586 T3466 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5459b2b main{StandardDirectoryReader(segments_6:3:nrt _0(5.0):c4)}
[junit4:junit4] 2> 1039589 T3437 C1010 P40036 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 1039588 T3420 C1014 P10710 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 82
[junit4:junit4] 2> 1039589 T3454 C1012 P20404 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 83
[junit4:junit4] 2> 1039593 T3400 C1011 P19850 oass.SolrIndexSearcher.<init> Opening Searcher@3cad59cf main
[junit4:junit4] 2> 1039593 T3400 C1011 P19850 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1039593 T3437 C1010 P40036 oass.SolrIndexSearcher.<init> Opening Searcher@681a4998 main
[junit4:junit4] 2> 1039594 T3412 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3cad59cf main{StandardDirectoryReader(segments_6:3:nrt _0(5.0):c4)}
[junit4:junit4] 2> 1039594 T3437 C1010 P40036 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1039595 T3400 C1011 P19850 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 89
[junit4:junit4] 2> 1039595 T3445 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@681a4998 main{StandardDirectoryReader(segments_5:3:nrt _0(5.0):c4)}
[junit4:junit4] 2> 1039596 T3437 C1010 P40036 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 90
[junit4:junit4] 2> 1039596 T3436 C1010 P40036 oasup.LogUpdateProcessor.finish [collection1] webapp=/h_ow/h path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 93
[junit4:junit4] 2> 1039597 T3368 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 1039598 T3368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1039599 T3368 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 1039602 T3401 C1011 P19850 oasc.SolrCore.execute [collection1] webapp=/h_ow/h path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 1039606 T3418 C1014 P10710 oasc.SolrCore.execute [collection1] webapp=/h_ow/h path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=2 status=0 QTime=0
[junit4:junit4] 2> 1039609 T3434 C1010 P40036 oasc.SolrCore.execute [collection1] webapp=/h_ow/h path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 1039613 T3451 C1012 P20404 oasc.SolrCore.execute [collection1] webapp=/h_ow/h path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 1039614 T3368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1039631 T3438 oasha.CollectionsHandler.handleSyncShardAction Syncing shard : shard=shard1&action=SYNCSHARD&collection=collection1&wt=javabin&version=2
[junit4:junit4] 2> 1039632 T3438 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1039644 T3483 oasha.CoreAdminHandler.handleRequestSyncAction I have been requested to sync up my shard
[junit4:junit4] 2> 1039644 T3483 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:19850/h_ow/h/collection1/
[junit4:junit4] 2> 1039645 T3483 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:19850/h_ow/h START replicas=[http://127.0.0.1:10710/h_ow/h/collection1/, http://127.0.0.1:40036/h_ow/h/collection1/, http://127.0.0.1:20404/h_ow/h/collection1/] nUpdates=100
[junit4:junit4] 2> 1039647 T3421 C1014 P10710 oasc.SolrCore.execute [collection1] webapp=/h_ow/h path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 1039647 T3455 C1012 P20404 oasc.SolrCore.execute [collection1] webapp=/h_ow/h path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 1039647 T3439 C1010 P40036 oasc.SolrCore.execute [collection1] webapp=/h_ow/h path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 1039648 T3483 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:19850/h_ow/h Received 3 versions from 127.0.0.1:10710/h_ow/h/collection1/
[junit4:junit4] 2> 1039649 T3483 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:19850/h_ow/h Our versions are newer. ourLowThreshold=1438305543259160576 otherHigh=1438305543292715008
[junit4:junit4] 2> 1039649 T3483 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:19850/h_ow/h Received 5 versions from 127.0.0.1:20404/h_ow/h/collection1/
[junit4:junit4] 2> 1039649 T3483 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:19850/h_ow/h Our versions are newer. ourLowThreshold=1438305543259160576 otherHigh=1438305543308443648
[junit4:junit4] 2> 1039650 T3483 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:19850/h_ow/h Received 5 versions from 127.0.0.1:40036/h_ow/h/collection1/
[junit4:junit4] 2> 1039650 T3483 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:19850/h_ow/h Our versions are newer. ourLowThreshold=1438305543259160576 otherHigh=1438305543308443648
[junit4:junit4] 2> 1039651 T3483 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:19850/h_ow/h DONE. sync succeeded
[junit4:junit4] 2> 1039651 T3483 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1039651 T3483 oasc.SyncStrategy.syncToMe http://127.0.0.1:19850/h_ow/h/collection1/: try and ask http://127.0.0.1:10710/h_ow/h/collection1/ to sync
[junit4:junit4] 2> 1039652 T3483 oasc.SyncStrategy.syncToMe http://127.0.0.1:19850/h_ow/h/collection1/: try and ask http://127.0.0.1:40036/h_ow/h/collection1/ to sync
[junit4:junit4] 2> 1039652 T3483 oasc.SyncStrategy.syncToMe http://127.0.0.1:19850/h_ow/h/collection1/: try and ask http://127.0.0.1:20404/h_ow/h/collection1/ to sync
[junit4:junit4] 2> 1039653 T3422 C1014 P10710 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:10710/h_ow/h START replicas=[http://127.0.0.1:19850/h_ow/h/collection1/] nUpdates=100
[junit4:junit4] 2> 1039654 T3490 C1010 P40036 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:40036/h_ow/h START replicas=[http://127.0.0.1:19850/h_ow/h/collection1/] nUpdates=100
[junit4:junit4] 2> 1039654 T3456 C1012 P20404 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:20404/h_ow/h START replicas=[http://127.0.0.1:19850/h_ow/h/collection1/] nUpdates=100
[junit4:junit4] 2> 1039656 T3494 C1011 P19850 oasc.SolrCore.execute [collection1] webapp=/h_ow/h path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 1039656 T3495 C1011 P19850 oasc.SolrCore.execute [collection1] webapp=/h_ow/h path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 1039656 T3490 C1010 P40036 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:40036/h_ow/h Received 5 versions from 127.0.0.1:19850/h_ow/h/collection1/
[junit4:junit4] 2> 1039656 T3402 C1011 P19850 oasc.SolrCore.execute [collection1] webapp=/h_ow/h path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 1039657 T3490 C1010 P40036 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:40036/h_ow/h Our versions are newer. ourLowThreshold=1438305543259160576 otherHigh=1438305543308443648
[junit4:junit4] 2> 1039657 T3456 C1012 P20404 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:20404/h_ow/h Received 5 versions from 127.0.0.1:19850/h_ow/h/collection1/
[junit4:junit4] 2> 1039658 T3490 C1010 P40036 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:40036/h_ow/h DONE. sync succeeded
[junit4:junit4] 2> 1039657 T3422 C1014 P10710 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:10710/h_ow/h Received 5 versions from 127.0.0.1:19850/h_ow/h/collection1/
[junit4:junit4] 2> 1039659 T3490 C1010 P40036 oasc.SolrCore.execute [collection1] webapp=/h_ow/h path=/get params={sync=http://127.0.0.1:19850/h_ow/h/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=5
[junit4:junit4] 2> 1039658 T3456 C1012 P20404 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:20404/h_ow/h Our versions are newer. ourLowThreshold=1438305543259160576 otherHigh=1438305543308443648
[junit4:junit4
[...truncated too long message...]
2> 2002974 T4307 oasu.PeerSync.handleResponse WARN PeerSync: core=collection1 url=http://127.0.0.1:51407 couldn't connect to http://127.0.0.1:26890/collection1/, counting as success
[junit4:junit4] 2> 2002974 T4307 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:51407 DONE. sync succeeded
[junit4:junit4] 2> 2002974 T4307 oasc.SyncStrategy.syncReplicas We have been closed, won't attempt to sync replicas back to leader
[junit4:junit4] 2> 2002975 T4307 oasc.ShardLeaderElectionContext.rejoinLeaderElection Not rejoining election because CoreContainer is shutdown
[junit4:junit4] 2> 2002975 T4307 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@6cccb931
[junit4:junit4] 2> 2002980 T4307 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=2,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=823,adds=823,deletesById=406,deletesByQuery=0,errors=0,cumulative_adds=865,cumulative_deletesById=406,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4:junit4] 2> 2002980 T4307 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 2002980 T4307 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 2002980 T4307 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 2002993 T4307 C1214 P51407 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@e53b2cd lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c72a0e1),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@e53b2cd lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c72a0e1),segFN=segments_5,generation=5}
[junit4:junit4] 2> 2002994 T4307 C1214 P51407 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 2002997 T4307 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 2002997 T4307 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 2002997 T4307 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ShardSplitTest-1371675766673/jetty4 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ShardSplitTest-1371675766673/jetty4;done=false>>]
[junit4:junit4] 2> 2002998 T4307 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ShardSplitTest-1371675766673/jetty4
[junit4:junit4] 2> 2002998 T4307 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ShardSplitTest-1371675766673/jetty4/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ShardSplitTest-1371675766673/jetty4/index;done=false>>]
[junit4:junit4] 2> 2002998 T4307 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ShardSplitTest-1371675766673/jetty4/index
[junit4:junit4] 2> 2002998 T4307 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 2002998 T4307 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2002998 T4307 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 2002999 T4307 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:235)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:232)
[junit4:junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:232)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4] 2>
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {id=Pulsing41(freqCutoff=1 minBlockSize=39 maxBlockSize=88), _version_=PostingsFormat(name=Memory doPackFST= true), n_ti=PostingsFormat(name=Memory doPackFST= true)}, docValues:{}, sim=DefaultSimilarity, locale=sl, timezone=Africa/Kigali
[junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=3,free=86041832,total=361758720
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TermsComponentTest, AnalysisAfterCoreReloadTest, TestBadConfig, StatelessScriptUpdateProcessorFactoryTest, TestReloadAndDeleteDocs, TestExtendedDismaxParser, TestValueSourceCache, ReturnFieldsTest, DirectUpdateHandlerOptimizeTest, JsonLoaderTest, TestReversedWildcardFilterFactory, SliceStateUpdateTest, SchemaVersionSpecificBehaviorTest, LegacyHTMLStripCharFilterTest, CSVRequestHandlerTest, TestDocumentBuilder, BinaryUpdateRequestHandlerTest, TestQueryUtils, QueryParsingTest, TestJoin, LoggingHandlerTest, BasicDistributedZkTest, SolrIndexSplitterTest, ZkControllerTest, CachingDirectoryFactoryTest, TestPluginEnable, TestSweetSpotSimilarityFactory, PreAnalyzedFieldTest, TestLMDirichletSimilarityFactory, TestSolrDeletionPolicy2, FileBasedSpellCheckerTest, TestPerFieldSimilarity, TestConfig, TestFaceting, ExternalFileFieldSortTest, LeaderElectionTest, UpdateRequestProcessorFactoryTest, PreAnalyzedUpdateProcessorTest, RegexBoostProcessorTest, PluginInfoTest, ScriptEngineTest, DateFieldTest, TestSolrQueryParser, TestPhraseSuggestions, ShowFileRequestHandlerTest, TestCloudManagedSchemaAddField, AlternateDirectoryTest, OverseerCollectionProcessorTest, CircularListTest, AliasIntegrationTest, TestIBSimilarityFactory, TestRemoteStreaming, TestReplicationHandler, TestSolrIndexConfig, FieldMutatingUpdateProcessorTest, DistributedQueryElevationComponentTest, SpatialFilterTest, SearchHandlerTest, SOLR749Test, TestSchemaSimilarityResource, TestPropInjectDefaults, ZkCLITest, ShardRoutingTest, TestRealTimeGet, TermVectorComponentTest, TestPseudoReturnFields, TestStressVersions, CacheHeaderTest, DocValuesTest, TestSolrCoreProperties, WordBreakSolrSpellCheckerTest, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest, SyncSliceTest, ShardRoutingCustomTest, AutoCommitTest, TestRecovery, TestHashPartitioner, DistributedSpellCheckComponentTest, TestReload, TestCoreContainer, SpellCheckComponentTest, QueryElevationComponentTest, PeerSyncTest, ConvertedLegacyTest, TestSort, BasicFunctionalityTest, SoftAutoCommitTest, TestCoreDiscovery, SuggesterFSTTest, TestFoldingMultitermQuery, SuggesterTest, TestTrie, PolyFieldTest, TestAtomicUpdateErrorCases, DistanceFunctionTest, SolrInfoMBeanTest, DebugComponentTest, TestSurroundQueryParser, StandardRequestHandlerTest, TestWriterPerf, PrimitiveFieldTypeTest, DocumentBuilderTest, FastVectorHighlighterTest, IndexSchemaTest, TestCollationField, JSONWriterTest, MBeansHandlerTest, TestPartialUpdateDeduplication, TestComponentsName, TestLFUCache, TestPropInject, UpdateParamsTest, CopyFieldTest, BadComponentTest, TestDocSet, TestBinaryField, TestSearchPerf, MinimalSchemaTest, OutputWriterTest, TestFuzzyAnalyzedSuggestions, TestPostingsSolrHighlighter, DirectSolrConnectionTest, NotRequiredUniqueKeyTest, TestLuceneMatchVersion, TestCharFilters, TestCodecSupport, SynonymTokenizerTest, TestXIncludeConfig, EchoParamsTest, TestDefaultSimilarityFactory, TestFastWriter, TimeZoneUtilsTest, TestFastOutputStream, URLClassifyProcessorTest, ChaosMonkeyNothingIsSafeTest, PrimUtilsTest, SpellingQueryConverterTest, RAMDirectoryFactoryTest, TestLRUCache, UUIDFieldTest, TestRTGBase, ShardSplitTest]
[junit4:junit4] Completed on J0 in 253.04s, 1 test, 1 error <<< FAILURES!
[...truncated 29 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:386: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:366: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:181: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:437: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1243: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:887: There were test failures: 297 suites, 1252 tests, 1 error, 1 failure, 13 ignored (7 assumptions)
Total time: 53 minutes 39 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4076 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4076/
3 tests failed.
REGRESSION: org.apache.solr.cloud.RecoveryZkTest.testDistribSearch
Error Message:
expected:<174> but was:<44>
Stack Trace:
java.lang.AssertionError: expected:<174> but was:<44>
at __randomizedtesting.SeedInfo.seed([8982A827897334FE:864263FFE2C54C2]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.RecoveryZkTest.doTest(RecoveryZkTest.java:108)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
REGRESSION: org.apache.solr.cloud.ShardSplitTest.testDistribSearch
Error Message:
Server at http://127.0.0.1:62691/l_qkp/r returned non ok status:500, message:Server Error
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:62691/l_qkp/r returned non ok status:500, message:Server Error
at __randomizedtesting.SeedInfo.seed([8982A827897334FE:864263FFE2C54C2]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
at org.apache.solr.cloud.ShardSplitTest.doTest(ShardSplitTest.java:150)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
REGRESSION: org.apache.solr.cloud.SyncSliceTest.testDistribSearch
Error Message:
shard1 is not consistent. Got 305 from http://127.0.0.1:45840/did/j/collection1lastClient and got 255 from http://127.0.0.1:22110/did/j/collection1
Stack Trace:
java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:45840/did/j/collection1lastClient and got 255 from http://127.0.0.1:22110/did/j/collection1
at __randomizedtesting.SeedInfo.seed([8982A827897334FE:864263FFE2C54C2]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9385 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ShardSplitTest
[junit4:junit4] 2> 170350 T575 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /l_qkp/r
[junit4:junit4] 2> 170357 T575 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./solrtest-ShardSplitTest-1371640863023
[junit4:junit4] 2> 170359 T575 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 170360 T576 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 170460 T575 oasc.ZkTestServer.run start zk server on port:30630
[junit4:junit4] 2> 170461 T575 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 170490 T582 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@26c788b6 name:ZooKeeperConnection Watcher:127.0.0.1:30630 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 170491 T575 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 170491 T575 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 170506 T575 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 170519 T584 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@71a80d42 name:ZooKeeperConnection Watcher:127.0.0.1:30630/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 170519 T575 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 170520 T575 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 170532 T575 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 170541 T575 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 170551 T575 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 170555 T575 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 170556 T575 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 170562 T575 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 170563 T575 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 170667 T575 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 170668 T575 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 170678 T575 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 170678 T575 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 170682 T575 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 170683 T575 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 170687 T575 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 170688 T575 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 170691 T575 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 170692 T575 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 170696 T575 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 170696 T575 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 170700 T575 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 170701 T575 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 170969 T575 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 170973 T575 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:49264
[junit4:junit4] 2> 170973 T575 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 170974 T575 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 170975 T575 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-controljetty-1371640863376
[junit4:junit4] 2> 170975 T575 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-controljetty-1371640863376/solr.xml
[junit4:junit4] 2> 170976 T575 oasc.CoreContainer.<init> New CoreContainer 662668002
[junit4:junit4] 2> 170976 T575 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-controljetty-1371640863376/'
[junit4:junit4] 2> 170977 T575 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-controljetty-1371640863376/'
[junit4:junit4] 2> 171104 T575 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 171105 T575 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 171106 T575 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 171106 T575 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 171107 T575 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 171107 T575 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 171108 T575 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 171109 T575 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 171109 T575 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 171110 T575 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 171127 T575 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 171128 T575 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:30630/solr
[junit4:junit4] 2> 171129 T575 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 171130 T575 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 171133 T595 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@588f285a name:ZooKeeperConnection Watcher:127.0.0.1:30630 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 171133 T575 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 171136 T575 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 171149 T575 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 171151 T597 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7dd5c870 name:ZooKeeperConnection Watcher:127.0.0.1:30630/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 171151 T575 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 171154 T575 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 171163 T575 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 171167 T575 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 171171 T575 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49264_l_qkp%2Fr
[junit4:junit4] 2> 171177 T575 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49264_l_qkp%2Fr
[junit4:junit4] 2> 171181 T575 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 171193 T575 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 171202 T575 oasc.Overseer.start Overseer (id=89891855599599619-127.0.0.1:49264_l_qkp%2Fr-n_0000000000) starting
[junit4:junit4] 2> 171211 T575 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 171224 T599 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 171225 T575 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 171228 T575 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 171230 T575 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 171233 T598 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 171237 T600 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 171237 T600 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 172737 T598 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 172738 T598 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49264_l_qkp%2Fr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49264/l_qkp/r"}
[junit4:junit4] 2> 172738 T598 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 172739 T598 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 172777 T597 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 173241 T600 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-controljetty-1371640863376/collection1
[junit4:junit4] 2> 173241 T600 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 173242 T600 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 173242 T600 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 173244 T600 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-controljetty-1371640863376/collection1/'
[junit4:junit4] 2> 173246 T600 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardSplitTest-controljetty-1371640863376/collection1/lib/README' to classloader
[junit4:junit4] 2> 173247 T600 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardSplitTest-controljetty-1371640863376/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 173299 T600 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 173371 T600 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 173473 T600 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 173480 T600 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 174054 T600 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 174057 T600 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 174058 T600 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 174063 T600 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 174092 T600 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 174092 T600 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-controljetty-1371640863376/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1371640863023/control/data/
[junit4:junit4] 2> 174092 T600 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4702b4ee
[junit4:junit4] 2> 174093 T600 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 174093 T600 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371640863023/control/data
[junit4:junit4] 2> 174093 T600 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1371640863023/control/data/index/
[junit4:junit4] 2> 174094 T600 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1371640863023/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 174094 T600 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371640863023/control/data/index
[junit4:junit4] 2> 174097 T600 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@606acc48 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4c19fd21),segFN=segments_1,generation=1}
[junit4:junit4] 2> 174098 T600 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 174100 T600 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 174100 T600 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 174101 T600 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 174101 T600 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 174102 T600 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 174102 T600 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 174102 T600 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 174103 T600 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 174103 T600 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 174114 T600 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 174119 T600 oass.SolrIndexSearcher.<init> Opening Searcher@2dad6a1b main
[junit4:junit4] 2> 174119 T600 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ShardSplitTest-1371640863023/control/data/tlog
[junit4:junit4] 2> 174120 T600 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 174120 T600 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 174124 T601 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2dad6a1b main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 174124 T600 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 174125 T600 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49264/l_qkp/r collection:control_collection shard:shard1
[junit4:junit4] 2> 174125 T600 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 174144 T600 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 174156 T600 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 174156 T600 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 174156 T600 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:49264/l_qkp/r/collection1/
[junit4:junit4] 2> 174156 T600 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 174157 T600 oasc.SyncStrategy.syncToMe http://127.0.0.1:49264/l_qkp/r/collection1/ has no replicas
[junit4:junit4] 2> 174157 T600 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:49264/l_qkp/r/collection1/
[junit4:junit4] 2> 174157 T600 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 174283 T598 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 174370 T597 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 174420 T600 oasc.ZkController.register We are http://127.0.0.1:49264/l_qkp/r/collection1/ and leader is http://127.0.0.1:49264/l_qkp/r/collection1/
[junit4:junit4] 2> 174420 T600 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49264/l_qkp/r
[junit4:junit4] 2> 174420 T600 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 174421 T600 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 174421 T600 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 174423 T600 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 174424 T575 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 174425 T575 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 174425 T575 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 174438 T575 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 174439 T575 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 174442 T604 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c0fbf40 name:ZooKeeperConnection Watcher:127.0.0.1:30630/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 174442 T575 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 174444 T575 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 174447 T575 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 174712 T575 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 174715 T575 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:62691
[junit4:junit4] 2> 174716 T575 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 174717 T575 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 174717 T575 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty1-1371640867112
[junit4:junit4] 2> 174718 T575 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty1-1371640867112/solr.xml
[junit4:junit4] 2> 174718 T575 oasc.CoreContainer.<init> New CoreContainer 1058488936
[junit4:junit4] 2> 174719 T575 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty1-1371640867112/'
[junit4:junit4] 2> 174719 T575 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty1-1371640867112/'
[junit4:junit4] 2> 174847 T575 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 174848 T575 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 174848 T575 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 174849 T575 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 174849 T575 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 174850 T575 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 174850 T575 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 174851 T575 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 174851 T575 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 174852 T575 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 174871 T575 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 174872 T575 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:30630/solr
[junit4:junit4] 2> 174872 T575 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 174873 T575 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 174877 T615 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@28f67ca7 name:ZooKeeperConnection Watcher:127.0.0.1:30630 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 174877 T575 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 174904 T575 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 174913 T575 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 174924 T617 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6d922504 name:ZooKeeperConnection Watcher:127.0.0.1:30630/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 174924 T575 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 174941 T575 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 175901 T598 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 175902 T598 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:49264_l_qkp%2Fr_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49264_l_qkp%2Fr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49264/l_qkp/r"}
[junit4:junit4] 2> 175943 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 175943 T597 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 175943 T604 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 175945 T575 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:62691_l_qkp%2Fr
[junit4:junit4] 2> 175947 T575 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:62691_l_qkp%2Fr
[junit4:junit4] 2> 175957 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 175958 T604 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 175958 T597 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 175958 T604 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 175958 T597 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 175959 T617 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 175966 T618 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 175966 T618 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 177449 T598 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 177450 T598 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:62691_l_qkp%2Fr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:62691/l_qkp/r"}
[junit4:junit4] 2> 177450 T598 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 177451 T598 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 177456 T604 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 177456 T597 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 177456 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 177970 T618 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty1-1371640867112/collection1
[junit4:junit4] 2> 177970 T618 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 177971 T618 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 177971 T618 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 177973 T618 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty1-1371640867112/collection1/'
[junit4:junit4] 2> 177974 T618 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardSplitTest-jetty1-1371640867112/collection1/lib/README' to classloader
[junit4:junit4] 2> 177975 T618 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardSplitTest-jetty1-1371640867112/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 178028 T618 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 178099 T618 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 178200 T618 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 178207 T618 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 178779 T618 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 178784 T618 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 178786 T618 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 178792 T618 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 178827 T618 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 178827 T618 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty1-1371640867112/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty1/
[junit4:junit4] 2> 178828 T618 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4702b4ee
[junit4:junit4] 2> 178828 T618 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 178829 T618 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty1
[junit4:junit4] 2> 178829 T618 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty1/index/
[junit4:junit4] 2> 178830 T618 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 178830 T618 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty1/index
[junit4:junit4] 2> 178835 T618 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@784b47c8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@36e74c70),segFN=segments_1,generation=1}
[junit4:junit4] 2> 178835 T618 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 178839 T618 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 178839 T618 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 178840 T618 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 178841 T618 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 178842 T618 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 178842 T618 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 178842 T618 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 178843 T618 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 178844 T618 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 178860 T618 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 178868 T618 oass.SolrIndexSearcher.<init> Opening Searcher@926028a main
[junit4:junit4] 2> 178868 T618 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty1/tlog
[junit4:junit4] 2> 178869 T618 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 178870 T618 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 178875 T619 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@926028a main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 178878 T618 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 178879 T618 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:62691/l_qkp/r collection:collection1 shard:shard1
[junit4:junit4] 2> 178880 T618 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 178897 T618 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 178908 T618 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 178908 T618 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 178909 T618 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:62691/l_qkp/r/collection1/
[junit4:junit4] 2> 178909 T618 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 178909 T618 oasc.SyncStrategy.syncToMe http://127.0.0.1:62691/l_qkp/r/collection1/ has no replicas
[junit4:junit4] 2> 178910 T618 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:62691/l_qkp/r/collection1/
[junit4:junit4] 2> 178910 T618 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 178968 T598 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 178983 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 178983 T597 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 178983 T604 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 179027 T618 oasc.ZkController.register We are http://127.0.0.1:62691/l_qkp/r/collection1/ and leader is http://127.0.0.1:62691/l_qkp/r/collection1/
[junit4:junit4] 2> 179027 T618 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:62691/l_qkp/r
[junit4:junit4] 2> 179027 T618 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 179028 T618 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 179028 T618 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 179030 T618 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 179032 T575 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 179033 T575 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 179033 T575 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 179325 T575 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 179329 T575 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:57938
[junit4:junit4] 2> 179330 T575 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 179330 T575 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 179331 T575 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty2-1371640871717
[junit4:junit4] 2> 179332 T575 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty2-1371640871717/solr.xml
[junit4:junit4] 2> 179332 T575 oasc.CoreContainer.<init> New CoreContainer 434343587
[junit4:junit4] 2> 179333 T575 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty2-1371640871717/'
[junit4:junit4] 2> 179333 T575 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty2-1371640871717/'
[junit4:junit4] 2> 179462 T575 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 179463 T575 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 179463 T575 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 179464 T575 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 179464 T575 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 179465 T575 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 179466 T575 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 179466 T575 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 179467 T575 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 179467 T575 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 179486 T575 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 179487 T575 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:30630/solr
[junit4:junit4] 2> 179487 T575 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 179488 T575 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 179501 T631 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4c42cf93 name:ZooKeeperConnection Watcher:127.0.0.1:30630 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 179502 T575 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 179571 T575 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 179585 T575 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 179587 T633 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@71259875 name:ZooKeeperConnection Watcher:127.0.0.1:30630/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 179588 T575 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 179621 T575 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 180491 T598 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 180492 T598 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:62691_l_qkp%2Fr_collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:62691_l_qkp%2Fr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:62691/l_qkp/r"}
[junit4:junit4] 2> 180505 T604 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 180505 T633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 180505 T597 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 180505 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 180625 T575 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57938_l_qkp%2Fr
[junit4:junit4] 2> 180630 T575 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57938_l_qkp%2Fr
[junit4:junit4] 2> 180632 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 180632 T633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 180633 T597 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 180633 T604 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 180633 T597 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 180633 T604 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 180634 T617 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 180634 T633 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 180648 T634 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 180648 T634 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 182011 T598 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 182012 T598 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:57938_l_qkp%2Fr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:57938/l_qkp/r"}
[junit4:junit4] 2> 182013 T598 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 182013 T598 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 182018 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 182018 T597 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 182018 T633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 182018 T604 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 182652 T634 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty2-1371640871717/collection1
[junit4:junit4] 2> 182652 T634 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 182653 T634 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 182654 T634 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 182655 T634 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty2-1371640871717/collection1/'
[junit4:junit4] 2> 182657 T634 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardSplitTest-jetty2-1371640871717/collection1/lib/README' to classloader
[junit4:junit4] 2> 182658 T634 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardSplitTest-jetty2-1371640871717/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 182710 T634 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 182771 T634 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 182872 T634 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 182879 T634 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 183465 T634 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 183469 T634 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 183471 T634 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 183477 T634 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 183512 T634 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 183512 T634 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty2-1371640871717/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty2/
[junit4:junit4] 2> 183512 T634 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4702b4ee
[junit4:junit4] 2> 183513 T634 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 183514 T634 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty2
[junit4:junit4] 2> 183515 T634 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty2/index/
[junit4:junit4] 2> 183515 T634 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 183516 T634 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty2/index
[junit4:junit4] 2> 183520 T634 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1fcbdb9a lockFactory=org.apache.lucene.store.NativeFSLockFactory@3e7522a1),segFN=segments_1,generation=1}
[junit4:junit4] 2> 183520 T634 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 183524 T634 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 183524 T634 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 183525 T634 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 183526 T634 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 183527 T634 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 183527 T634 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 183528 T634 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 183528 T634 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 183529 T634 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 183546 T634 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 183554 T634 oass.SolrIndexSearcher.<init> Opening Searcher@320f0ea0 main
[junit4:junit4] 2> 183554 T634 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty2/tlog
[junit4:junit4] 2> 183555 T634 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 183556 T634 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 183562 T635 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@320f0ea0 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 183565 T634 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 183565 T634 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57938/l_qkp/r collection:collection1 shard:shard2
[junit4:junit4] 2> 183566 T634 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 183586 T634 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 183590 T634 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 183590 T634 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 183590 T634 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:57938/l_qkp/r/collection1/
[junit4:junit4] 2> 183591 T634 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 183591 T634 oasc.SyncStrategy.syncToMe http://127.0.0.1:57938/l_qkp/r/collection1/ has no replicas
[junit4:junit4] 2> 183591 T634 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:57938/l_qkp/r/collection1/
[junit4:junit4] 2> 183592 T634 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 185026 T598 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 185036 T604 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 185036 T597 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 185036 T633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 185036 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 185077 T634 oasc.ZkController.register We are http://127.0.0.1:57938/l_qkp/r/collection1/ and leader is http://127.0.0.1:57938/l_qkp/r/collection1/
[junit4:junit4] 2> 185077 T634 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57938/l_qkp/r
[junit4:junit4] 2> 185077 T634 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 185078 T634 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 185078 T634 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 185080 T634 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 185082 T575 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 185083 T575 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 185084 T575 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 185360 T575 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 185364 T575 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:12539
[junit4:junit4] 2> 185365 T575 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 185365 T575 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 185366 T575 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty3-1371640877766
[junit4:junit4] 2> 185367 T575 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty3-1371640877766/solr.xml
[junit4:junit4] 2> 185367 T575 oasc.CoreContainer.<init> New CoreContainer 1709006452
[junit4:junit4] 2> 185368 T575 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty3-1371640877766/'
[junit4:junit4] 2> 185368 T575 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty3-1371640877766/'
[junit4:junit4] 2> 185495 T575 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 185496 T575 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 185497 T575 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 185498 T575 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 185498 T575 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 185499 T575 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 185499 T575 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 185500 T575 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 185500 T575 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 185501 T575 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 185518 T575 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 185519 T575 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:30630/solr
[junit4:junit4] 2> 185519 T575 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 185521 T575 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 185524 T647 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@55432a38 name:ZooKeeperConnection Watcher:127.0.0.1:30630 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 185525 T575 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 185543 T575 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 185555 T575 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 185557 T649 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@37b49805 name:ZooKeeperConnection Watcher:127.0.0.1:30630/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 185557 T575 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 185563 T575 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 186543 T598 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 186544 T598 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:57938_l_qkp%2Fr_collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:57938_l_qkp%2Fr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:57938/l_qkp/r"}
[junit4:junit4] 2> 186549 T604 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 186549 T597 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 186549 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 186549 T649 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 186549 T633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 186567 T575 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:12539_l_qkp%2Fr
[junit4:junit4] 2> 186569 T575 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:12539_l_qkp%2Fr
[junit4:junit4] 2> 186571 T633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 186571 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 186572 T597 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 186572 T604 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 186572 T597 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 186572 T649 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 186573 T604 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 186573 T649 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 186573 T633 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 186574 T617 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 186580 T650 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 186580 T650 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 188055 T598 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 188056 T598 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:12539_l_qkp%2Fr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:12539/l_qkp/r"}
[junit4:junit4] 2> 188056 T598 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 188057 T598 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 188076 T604 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 188076 T649 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 188076 T597 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 188076 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 188076 T633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 188584 T650 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty3-1371640877766/collection1
[junit4:junit4] 2> 188584 T650 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 188585 T650 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 188586 T650 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 188588 T650 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty3-1371640877766/collection1/'
[junit4:junit4] 2> 188589 T650 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardSplitTest-jetty3-1371640877766/collection1/lib/README' to classloader
[junit4:junit4] 2> 188590 T650 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardSplitTest-jetty3-1371640877766/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 188642 T650 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 188703 T650 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 188804 T650 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 188811 T650 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 189402 T650 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 189406 T650 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 189409 T650 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 189414 T650 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 189449 T650 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 189450 T650 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty3-1371640877766/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty3/
[junit4:junit4] 2> 189450 T650 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4702b4ee
[junit4:junit4] 2> 189451 T650 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 189452 T650 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty3
[junit4:junit4] 2> 189452 T650 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty3/index/
[junit4:junit4] 2> 189452 T650 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 189453 T650 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty3/index
[junit4:junit4] 2> 189457 T650 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@71114a4b lockFactory=org.apache.lucene.store.NativeFSLockFactory@69e94b3),segFN=segments_1,generation=1}
[junit4:junit4] 2> 189457 T650 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 189461 T650 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 189461 T650 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 189462 T650 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 189463 T650 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 189464 T650 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 189464 T650 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 189465 T650 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 189466 T650 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 189466 T650 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 189483 T650 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 189491 T650 oass.SolrIndexSearcher.<init> Opening Searcher@148a5e75 main
[junit4:junit4] 2> 189492 T650 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty3/tlog
[junit4:junit4] 2> 189493 T650 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 189494 T650 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 189500 T651 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@148a5e75 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 189503 T650 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 189503 T650 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:12539/l_qkp/r collection:collection1 shard:shard1
[junit4:junit4] 2> 189507 T650 oasc.ZkController.register We are http://127.0.0.1:12539/l_qkp/r/collection1/ and leader is http://127.0.0.1:62691/l_qkp/r/collection1/
[junit4:junit4] 2> 189507 T650 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:12539/l_qkp/r
[junit4:junit4] 2> 189508 T650 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 189508 T650 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C119 name=collection1 org.apache.solr.core.SolrCore@53ab68de url=http://127.0.0.1:12539/l_qkp/r/collection1 node=127.0.0.1:12539_l_qkp%2Fr C119_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:12539_l_qkp%2Fr, base_url=http://127.0.0.1:12539/l_qkp/r}
[junit4:junit4] 2> 189508 T652 C119 P12539 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 189509 T650 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 189509 T652 C119 P12539 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 189510 T652 C119 P12539 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 189510 T652 C119 P12539 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 189511 T575 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 189512 T575 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 189512 T652 C119 P12539 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 189512 T575 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 189541 T607 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:12539_l_qkp%2Fr_collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 189582 T598 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 189583 T598 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:12539_l_qkp%2Fr_collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:12539_l_qkp%2Fr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:12539/l_qkp/r"}
[junit4:junit4] 2> 189596 T604 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 189596 T597 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 189596 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 189596 T649 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 189596 T633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 189836 T575 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 189839 T575 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:41032
[junit4:junit4] 2> 189840 T575 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 189841 T575 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 189841 T575 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty4-1371640882204
[junit4:junit4] 2> 189842 T575 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty4-1371640882204/solr.xml
[junit4:junit4] 2> 189842 T575 oasc.CoreContainer.<init> New CoreContainer 1856178865
[junit4:junit4] 2> 189843 T575 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty4-1371640882204/'
[junit4:junit4] 2> 189844 T575 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty4-1371640882204/'
[junit4:junit4] 2> 189990 T575 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 189991 T575 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 189992 T575 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 189992 T575 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 189993 T575 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 189993 T575 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 189994 T575 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 189995 T575 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 189995 T575 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 189996 T575 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 190016 T575 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 190017 T575 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:30630/solr
[junit4:junit4] 2> 190018 T575 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 190020 T575 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 190049 T664 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@365f4402 name:ZooKeeperConnection Watcher:127.0.0.1:30630 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 190050 T575 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 190060 T575 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 190073 T575 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 190076 T666 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9017643 name:ZooKeeperConnection Watcher:127.0.0.1:30630/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 190076 T575 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 190088 T575 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 190543 T607 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:12539_l_qkp%2Fr_collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 190544 T607 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:12539_l_qkp%252Fr_collection1&state=recovering&nodeName=127.0.0.1:12539_l_qkp%252Fr&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1002
[junit4:junit4] 2> 191092 T575 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41032_l_qkp%2Fr
[junit4:junit4] 2> 191094 T575 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:41032_l_qkp%2Fr
[junit4:junit4] 2> 191097 T633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 191098 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 191098 T649 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 191098 T604 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 191098 T597 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 191098 T666 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 191099 T597 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 191099 T604 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 191099 T649 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 191099 T633 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 191107 T617 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 191111 T667 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 191112 T667 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> C119_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:12539_l_qkp%2Fr, base_url=http://127.0.0.1:12539/l_qkp/r}
[junit4:junit4] 2> 192546 T652 C119 P12539 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:62691/l_qkp/r/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 192546 T652 C119 P12539 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:12539/l_qkp/r START replicas=[http://127.0.0.1:62691/l_qkp/r/collection1/] nUpdates=100
[junit4:junit4] 2> 192547 T652 C119 P12539 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 192548 T652 C119 P12539 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 192548 T652 C119 P12539 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 192548 T652 C119 P12539 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 192549 T652 C119 P12539 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 192549 T652 C119 P12539 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:62691/l_qkp/r/collection1/. core=collection1
[junit4:junit4] 2> 192550 T652 C119 P12539 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C120 name=collection1 org.apache.solr.core.SolrCore@78433998 url=http://127.0.0.1:62691/l_qkp/r/collection1 node=127.0.0.1:62691_l_qkp%2Fr C120_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:62691_l_qkp%2Fr, base_url=http://127.0.0.1:62691/l_qkp/r, leader=true}
[junit4:junit4] 2> 192558 T608 C120 P62691 oasc.SolrCore.execute [collection1] webapp=/l_qkp/r path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 192565 T609 C120 P62691 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 192568 T609 C120 P62691 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@784b47c8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@36e74c70),segFN=segments_1,generation=1}
[junit4:junit4] 2> 192569 T609 C120 P62691 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 192570 T609 C120 P62691 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@784b47c8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@36e74c70),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@784b47c8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@36e74c70),segFN=segments_2,generation=2}
[junit4:junit4] 2> 192570 T609 C120 P62691 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 192571 T609 C120 P62691 oass.SolrIndexSearcher.<init> Opening Searcher@44883a19 realtime
[junit4:junit4] 2> 192572 T609 C120 P62691 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 192572 T609 C120 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 192573 T652 C119 P12539 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 192574 T652 C119 P12539 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 192576 T610 C120 P62691 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 192577 T610 C120 P62691 oasc.SolrCore.execute [collection1] webapp=/l_qkp/r path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2
[junit4:junit4] 2> 192578 T652 C119 P12539 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 192578 T652 C119 P12539 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 192578 T652 C119 P12539 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 192582 T610 C120 P62691 oasc.SolrCore.execute [collection1] webapp=/l_qkp/r path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1
[junit4:junit4] 2> 192583 T652 C119 P12539 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 192584 T652 C119 P12539 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty3/index.20130620000625248
[junit4:junit4] 2> 192584 T652 C119 P12539 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@3c9d2cb1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b95aa1d) fullCopy=false
[junit4:junit4] 2> 192588 T610 C120 P62691 oasc.SolrCore.execute [collection1] webapp=/l_qkp/r path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 192591 T652 C119 P12539 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 192592 T652 C119 P12539 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 192592 T652 C119 P12539 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 192595 T652 C119 P12539 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@71114a4b lockFactory=org.apache.lucene.store.NativeFSLockFactory@69e94b3),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@71114a4b lockFactory=org.apache.lucene.store.NativeFSLockFactory@69e94b3),segFN=segments_2,generation=2}
[junit4:junit4] 2> 192596 T652 C119 P12539 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 192596 T652 C119 P12539 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 192596 T652 C119 P12539 oass.SolrIndexSearcher.<init> Opening Searcher@b8c56f6 main
[junit4:junit4] 2> 192598 T651 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@b8c56f6 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 192599 T652 C119 P12539 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty3/index.20130620000625248 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty3/index.20130620000625248;done=true>>]
[junit4:junit4] 2> 192599 T652 C119 P12539 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty3/index.20130620000625248
[junit4:junit4] 2> 192599 T652 C119 P12539 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty3/index.20130620000625248
[junit4:junit4] 2> 192600 T652 C119 P12539 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 192600 T652 C119 P12539 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 192600 T652 C119 P12539 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 192601 T652 C119 P12539 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 192603 T652 C119 P12539 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 192611 T598 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 192612 T598 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:41032_l_qkp%2Fr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:41032/l_qkp/r"}
[junit4:junit4] 2> 192612 T598 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 192612 T598 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 192618 T598 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:12539_l_qkp%2Fr_collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:12539_l_qkp%2Fr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:12539/l_qkp/r"}
[junit4:junit4] 2> 192629 T666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 192629 T597 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 192629 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 192629 T649 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 192629 T633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 192629 T604 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 193115 T667 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty4-1371640882204/collection1
[junit4:junit4] 2> 193115 T667 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 193116 T667 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 193117 T667 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 193119 T667 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty4-1371640882204/collection1/'
[junit4:junit4] 2> 193120 T667 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardSplitTest-jetty4-1371640882204/collection1/lib/README' to classloader
[junit4:junit4] 2> 193121 T667 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardSplitTest-jetty4-1371640882204/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 193181 T667 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 193251 T667 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 193253 T667 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 193272 T667 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 193917 T667 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 193922 T667 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 193924 T667 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 193930 T667 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 193963 T667 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 193964 T667 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty4-1371640882204/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty4/
[junit4:junit4] 2> 193964 T667 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4702b4ee
[junit4:junit4] 2> 193965 T667 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 193966 T667 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty4
[junit4:junit4] 2> 193966 T667 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty4/index/
[junit4:junit4] 2> 193966 T667 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 193967 T667 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty4/index
[junit4:junit4] 2> 193972 T667 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5c396b66 lockFactory=org.apache.lucene.store.NativeFSLockFactory@c6df8a4),segFN=segments_1,generation=1}
[junit4:junit4] 2> 193972 T667 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 193976 T667 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 193976 T667 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 193977 T667 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 193978 T667 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 193979 T667 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 193979 T667 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 193980 T667 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 193981 T667 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 193982 T667 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 194001 T667 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 194010 T667 oass.SolrIndexSearcher.<init> Opening Searcher@4bce8bc5 main
[junit4:junit4] 2> 194011 T667 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty4/tlog
[junit4:junit4] 2> 194012 T667 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 194012 T667 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 194018 T670 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4bce8bc5 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 194021 T667 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 194021 T667 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:41032/l_qkp/r collection:collection1 shard:shard2
[junit4:junit4] 2> 194026 T667 oasc.ZkController.register We are http://127.0.0.1:41032/l_qkp/r/collection1/ and leader is http://127.0.0.1:57938/l_qkp/r/collection1/
[junit4:junit4] 2> 194026 T667 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:41032/l_qkp/r
[junit4:junit4] 2> 194026 T667 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 194027 T667 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C121 name=collection1 org.apache.solr.core.SolrCore@55a186a5 url=http://127.0.0.1:41032/l_qkp/r/collection1 node=127.0.0.1:41032_l_qkp%2Fr C121_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:41032_l_qkp%2Fr, base_url=http://127.0.0.1:41032/l_qkp/r}
[junit4:junit4] 2> 194027 T671 C121 P41032 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 194028 T667 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 194028 T671 C121 P41032 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 194028 T671 C121 P41032 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 194029 T671 C121 P41032 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 194029 T575 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 194030 T575 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 194030 T671 C121 P41032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 194031 T575 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 194055 T575 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 194056 T623 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:41032_l_qkp%2Fr_collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 194057 T575 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 194058 T575 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 194059 T575 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 194143 T598 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 194144 T598 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:41032_l_qkp%2Fr_collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:41032_l_qkp%2Fr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:41032/l_qkp/r"}
[junit4:junit4] 2> 194158 T604 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 194158 T649 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 194158 T597 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 194158 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 194158 T666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 194158 T633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 195058 T623 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:41032_l_qkp%2Fr_collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 195058 T623 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:41032_l_qkp%252Fr_collection1&state=recovering&nodeName=127.0.0.1:41032_l_qkp%252Fr&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1002
[junit4:junit4] 2> 195061 T575 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 196064 T575 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C121_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:41032_l_qkp%2Fr, base_url=http://127.0.0.1:41032/l_qkp/r}
[junit4:junit4] 2> 197061 T671 C121 P41032 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:57938/l_qkp/r/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 197061 T671 C121 P41032 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:41032/l_qkp/r START replicas=[http://127.0.0.1:57938/l_qkp/r/collection1/] nUpdates=100
[junit4:junit4] 2> 197062 T671 C121 P41032 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 197062 T671 C121 P41032 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 197062 T671 C121 P41032 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 197063 T671 C121 P41032 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 197063 T671 C121 P41032 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 197063 T671 C121 P41032 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:57938/l_qkp/r/collection1/. core=collection1
[junit4:junit4] 2> 197064 T671 C121 P41032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 197067 T575 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C122 name=collection1 org.apache.solr.core.SolrCore@426a6f70 url=http://127.0.0.1:57938/l_qkp/r/collection1 node=127.0.0.1:57938_l_qkp%2Fr C122_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:57938_l_qkp%2Fr, base_url=http://127.0.0.1:57938/l_qkp/r, leader=true}
[junit4:junit4] 2> 197072 T624 C122 P57938 oasc.SolrCore.execute [collection1] webapp=/l_qkp/r path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 197078 T625 C122 P57938 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 197081 T625 C122 P57938 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1fcbdb9a lockFactory=org.apache.lucene.store.NativeFSLockFactory@3e7522a1),segFN=segments_1,generation=1}
[junit4:junit4] 2> 197082 T625 C122 P57938 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 197083 T625 C122 P57938 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1fcbdb9a lockFactory=org.apache.lucene.store.NativeFSLockFactory@3e7522a1),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1fcbdb9a lockFactory=org.apache.lucene.store.NativeFSLockFactory@3e7522a1),segFN=segments_2,generation=2}
[junit4:junit4] 2> 197083 T625 C122 P57938 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 197084 T625 C122 P57938 oass.SolrIndexSearcher.<init> Opening Searcher@6a750a44 realtime
[junit4:junit4] 2> 197084 T625 C122 P57938 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 197085 T625 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 197086 T671 C121 P41032 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 197087 T671 C121 P41032 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 197089 T626 C122 P57938 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 197089 T626 C122 P57938 oasc.SolrCore.execute [collection1] webapp=/l_qkp/r path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 197090 T671 C121 P41032 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 197091 T671 C121 P41032 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 197091 T671 C121 P41032 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 197095 T626 C122 P57938 oasc.SolrCore.execute [collection1] webapp=/l_qkp/r path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1
[junit4:junit4] 2> 197096 T671 C121 P41032 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 197097 T671 C121 P41032 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty4/index.20130620000629761
[junit4:junit4] 2> 197097 T671 C121 P41032 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@2de4ba27 lockFactory=org.apache.lucene.store.NativeFSLockFactory@688265a7) fullCopy=false
[junit4:junit4] 2> 197101 T626 C122 P57938 oasc.SolrCore.execute [collection1] webapp=/l_qkp/r path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 197104 T671 C121 P41032 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 197105 T671 C121 P41032 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 197105 T671 C121 P41032 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 197108 T671 C121 P41032 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5c396b66 lockFactory=org.apache.lucene.store.NativeFSLockFactory@c6df8a4),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5c396b66 lockFactory=org.apache.lucene.store.NativeFSLockFactory@c6df8a4),segFN=segments_2,generation=2}
[junit4:junit4] 2> 197108 T671 C121 P41032 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 197109 T671 C121 P41032 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 197109 T671 C121 P41032 oass.SolrIndexSearcher.<init> Opening Searcher@44d58377 main
[junit4:junit4] 2> 197110 T670 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@44d58377 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 197111 T671 C121 P41032 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty4/index.20130620000629761 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty4/index.20130620000629761;done=true>>]
[junit4:junit4] 2> 197112 T671 C121 P41032 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty4/index.20130620000629761
[junit4:junit4] 2> 197112 T671 C121 P41032 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ShardSplitTest-1371640863023/jetty4/index.20130620000629761
[junit4:junit4] 2> 197112 T671 C121 P41032 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 197112 T671 C121 P41032 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 197113 T671 C121 P41032 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 197113 T671 C121 P41032 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 197115 T671 C121 P41032 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 197166 T598 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 197167 T598 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:41032_l_qkp%2Fr_collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:41032_l_qkp%2Fr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:41032/l_qkp/r"}
[junit4:junit4] 2> 197180 T666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 197180 T633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 197181 T597 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 197181 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 197180 T649 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 197180 T604 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 198070 T575 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 198072 T575 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C123 name=collection1 org.apache.solr.core.SolrCore@74028703 url=http://127.0.0.1:49264/l_qkp/r/collection1 node=127.0.0.1:49264_l_qkp%2Fr C123_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:49264_l_qkp%2Fr, base_url=http://127.0.0.1:49264/l_qkp/r, leader=true}
[junit4:junit4] 2> 198083 T587 C123 P49264 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 198086 T587 C123 P49264 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@606acc48 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4c19fd21),segFN=segments_1,generation=1}
[junit4:junit4] 2> 198087 T587 C123 P49264 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 198088 T587 C123 P49264 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@606acc48 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4c19fd21),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@606acc48 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4c19fd21),segFN=segments_2,generation=2}
[junit4:junit4] 2> 198089 T587 C123 P49264 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 198089 T587 C123 P49264 oass.SolrIndexSearcher.<init> Opening Searcher@e7ef9b0 main
[junit4:junit4] 2> 198090 T587 C123 P49264 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 198091 T601 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@e7ef9b0 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 198092 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4] 2> ASYNC NEW_CORE C124 name=collection1 org.apache.solr.core.SolrCore@78433998 url=http://127.0.0.1:62691/l_qkp/r/collection1 node=127.0.0.1:62691_l_qkp%2Fr C124_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:62691_l_qkp%2Fr, base_url=http://127.0.0.1:62691/l_qkp/r, leader=true}
[junit4:junit4] 2> 198106 T611 C124 P62691 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:62691/l_qkp/r/collection1/, StdNode: http://127.0.0.1:12539/l_qkp/r/collection1/, StdNode: http://127.0.0.1:57938/l_qkp/r/collection1/, StdNode: http://127.0.0.1:41032/l_qkp/r/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 198110 T612 C124 P62691 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 198110 T627 C122 P57938 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 198112 T612 C124 P62691 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@784b47c8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@36e74c70),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@784b47c8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@36e74c70),segFN=segments_3,generation=3}
[junit4:junit4] 2> 198112 T627 C122 P57938 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1fcbdb9a lockFactory=org.apache.lucene.store.NativeFSLockFactory@3e7522a1),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1fcbdb9a lockFactory=org.apache.lucene.store.NativeFSLockFactory@3e7522a1),segFN=segments_3,generation=3}
[junit4:junit4] 2> 198112 T612 C124 P62691 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 198113 T627 C122 P57938 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 198114 T612 C124 P62691 oass.SolrIndexSearcher.<init> Opening Searcher@5983e83b main
[junit4:junit4] 2> 198114 T627 C122 P57938 oass.SolrIndexSearcher.<init> Opening Searcher@4e34c53 main
[junit4:junit4] 2> 198115 T612 C124 P62691 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 198115 T627 C122 P57938 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 198116 T619 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5983e83b main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 198116 T635 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4e34c53 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 198117 T612 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 198117 T627 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> C121_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:41032_l_qkp%2Fr, base_url=http://127.0.0.1:41032/l_qkp/r}
[junit4:junit4] 2> 198119 T656 C121 P41032 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C125 name=collection1 org.apache.solr.core.SolrCore@53ab68de url=http://127.0.0.1:12539/l_qkp/r/collection1 node=127.0.0.1:12539_l_qkp%2Fr C125_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:12539_l_qkp%2Fr, base_url=http://127.0.0.1:12539/l_qkp/r}
[junit4:junit4] 2> 198119 T639 C125 P12539 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 198122 T656 C121 P41032 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5c396b66 lockFactory=org.apache.lucene.store.NativeFSLockFactory@c6df8a4),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5c396b66 lockFactory=org.apache.lucene.store.NativeFSLockFactory@c6df8a4),segFN=segments_3,generation=3}
[junit4:junit4] 2> 198123 T639 C125 P12539 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@71114a4b lockFactory=org.apache.lucene.store.NativeFSLockFactory@69e94b3),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@71114a4b lockFactory=org.apache.lucene.store.NativeFSLockFactory@69e94b3),segFN=segments_3,generation=3}
[junit4:junit4] 2> 198123 T656 C121 P41032 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 198123 T639 C125 P12539 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 198124 T656 C121 P41032 oass.SolrIndexSearcher.<init> Opening Searcher@4619bd69 main
[junit4:junit4] 2> 198125 T639 C125 P12539 oass.SolrIndexSearcher.<init> Opening Searcher@49ea254c main
[junit4:junit4] 2> 198125 T656 C121 P41032 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 198126 T639 C125 P12539 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 198126 T670 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4619bd69 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 198127 T651 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@49ea254c main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 198127 T656 C121 P41032 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 8
[junit4:junit4] 2> 198128 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4] 2> 198129 T611 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 24
[junit4:junit4] 2> 198130 T575 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 198134 T613 C124 P62691 oasc.SolrCore.execute [collection1] webapp=/l_qkp/r path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 198138 T640 C125 P12539 oasc.SolrCore.execute [collection1] webapp=/l_qkp/r path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 198141 T628 C122 P57938 oasc.SolrCore.execute [collection1] webapp=/l_qkp/r path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 198144 T657 C121 P41032 oasc.SolrCore.execute [collection1] webapp=/l_qkp/r path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 200155 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1438269720827527168)} 0 7
[junit4:junit4] 2> 200167 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={update.distrib=FROMLEADER&_version_=-1438269720834867200&update.from=http://127.0.0.1:62691/l_qkp/r/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438269720834867200)} 0 2
[junit4:junit4] 2> 200173 T658 C121 P41032 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={update.distrib=FROMLEADER&_version_=-1438269720840110080&update.from=http://127.0.0.1:57938/l_qkp/r/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438269720840110080)} 0 2
[junit4:junit4] 2> 200174 T627 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1438269720840110080)} 0 9
[junit4:junit4] 2> 200175 T611 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1438269720834867200)} 0 16
[junit4:junit4] 2> 200180 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[a!0 (1438269720854790144)]} 0 2
[junit4:junit4] 2> 200192 T658 C121 P41032 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[a!0 (1438269720863178752)]} 0 2
[junit4:junit4] 2> 200193 T627 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[a!0 (1438269720863178752)]} 0 7
[junit4:junit4] 2> 200194 T611 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[a!0]} 0 11
[junit4:junit4] 2> 200198 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[b!1 (1438269720874713088)]} 0 1
[junit4:junit4] 2> 200207 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[b!1 (1438269720878907392)]} 0 2
[junit4:junit4] 2> 200208 T611 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[b!1 (1438269720878907392)]} 0 7
[junit4:junit4] 2> 200212 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[c!2 (1438269720889393152)]} 0 1
[junit4:junit4] 2> 200223 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[c!2 (1438269720895684608)]} 0 2
[junit4:junit4] 2> 200224 T607 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[c!2 (1438269720895684608)]} 0 7
[junit4:junit4] 2> 200225 T629 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[c!2]} 0 10
[junit4:junit4] 2> 200229 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[d!3 (1438269720907218944)]} 0 1
[junit4:junit4] 2> 200238 T658 C121 P41032 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[d!3 (1438269720912461824)]} 0 1
[junit4:junit4] 2> 200239 T629 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[d!3 (1438269720912461824)]} 0 6
[junit4:junit4] 2> 200243 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[e!4 (1438269720921899008)]} 0 1
[junit4:junit4] 2> 200252 T658 C121 P41032 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[e!4 (1438269720926093312)]} 0 2
[junit4:junit4] 2> 200253 T629 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[e!4 (1438269720926093312)]} 0 7
[junit4:junit4] 2> 200257 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[f!5 (1438269720936579072)]} 0 1
[junit4:junit4] 2> 200266 T658 C121 P41032 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[f!5 (1438269720941821952)]} 0 1
[junit4:junit4] 2> 200267 T629 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[f!5 (1438269720941821952)]} 0 6
[junit4:junit4] 2> 200271 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[g!6 (1438269720951259136)]} 0 1
[junit4:junit4] 2> 200282 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[g!6 (1438269720958599168)]} 0 1
[junit4:junit4] 2> 200283 T607 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[g!6 (1438269720958599168)]} 0 6
[junit4:junit4] 2> 200284 T629 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[g!6]} 0 10
[junit4:junit4] 2> 200288 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[h!7 (1438269720969084928)]} 0 1
[junit4:junit4] 2> 200296 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[h!7 (1438269720973279232)]} 0 1
[junit4:junit4] 2> 200297 T611 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[h!7 (1438269720973279232)]} 0 6
[junit4:junit4] 2> 200301 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[i!8 (1438269720982716416)]} 0 1
[junit4:junit4] 2> 200310 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[i!8 (1438269720986910720)]} 0 2
[junit4:junit4] 2> 200311 T611 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[i!8 (1438269720986910720)]} 0 7
[junit4:junit4] 2> 200315 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[j!9 (1438269720997396480)]} 0 1
[junit4:junit4] 2> 200328 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[j!9 (1438269721005785088)]} 0 2
[junit4:junit4] 2> 200329 T607 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[j!9 (1438269721005785088)]} 0 7
[junit4:junit4] 2> 200330 T629 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[j!9]} 0 11
[junit4:junit4] 2> 200335 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[k!10 (1438269721017319424)]} 0 2
[junit4:junit4] 2> 200346 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[k!10 (1438269721025708032)]} 0 1
[junit4:junit4] 2> 200347 T607 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[k!10 (1438269721025708032)]} 0 6
[junit4:junit4] 2> 200348 T629 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[k!10]} 0 10
[junit4:junit4] 2> 200352 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[l!11 (1438269721036193792)]} 0 1
[junit4:junit4] 2> 200359 T658 C121 P41032 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[l!11 (1438269721040388096)]} 0 1
[junit4:junit4] 2> 200360 T629 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[l!11 (1438269721040388096)]} 0 5
[junit4:junit4] 2> 200364 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[m!12 (1438269721048776704)]} 0 1
[junit4:junit4] 2> 200376 T658 C121 P41032 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[m!12 (1438269721057165312)]} 0 1
[junit4:junit4] 2> 200378 T627 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[m!12 (1438269721057165312)]} 0 7
[junit4:junit4] 2> 200378 T611 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[m!12]} 0 10
[junit4:junit4] 2> 200383 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[n!13 (1438269721068699648)]} 0 1
[junit4:junit4] 2> 200392 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[n!13 (1438269721073942528)]} 0 1
[junit4:junit4] 2> 200393 T611 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[n!13 (1438269721073942528)]} 0 6
[junit4:junit4] 2> 200398 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[o!14 (1438269721084428288)]} 0 1
[junit4:junit4] 2> 200411 T658 C121 P41032 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[o!14 (1438269721092816896)]} 0 2
[junit4:junit4] 2> 200412 T627 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[o!14 (1438269721092816896)]} 0 7
[junit4:junit4] 2> 200412 T611 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[o!14]} 0 10
[junit4:junit4] 2> 200417 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[p!15 (1438269721104351232)]} 0 1
[junit4:junit4] 2> 200429 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[p!15 (1438269721112739840)]} 0 1
[junit4:junit4] 2> 200430 T607 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[p!15 (1438269721112739840)]} 0 6
[junit4:junit4] 2> 200431 T629 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[p!15]} 0 10
[junit4:junit4] 2> 200435 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[q!16 (1438269721123225600)]} 0 1
[junit4:junit4] 2> 200447 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[q!16 (1438269721131614208)]} 0 1
[junit4:junit4] 2> 200448 T607 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[q!16 (1438269721131614208)]} 0 6
[junit4:junit4] 2> 200449 T629 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[q!16]} 0 10
[junit4:junit4] 2> 200454 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[r!17 (1438269721143148544)]} 0 1
[junit4:junit4] 2> 200466 T658 C121 P41032 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[r!17 (1438269721151537152)]} 0 1
[junit4:junit4] 2> 200467 T627 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[r!17 (1438269721151537152)]} 0 6
[junit4:junit4] 2> 200468 T611 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[r!17]} 0 10
[junit4:junit4] 2> 200472 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[s!18 (1438269721162022912)]} 0 1
[junit4:junit4] 2> 200481 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[s!18 (1438269721167265792)]} 0 1
[junit4:junit4] 2> 200482 T611 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[s!18 (1438269721167265792)]} 0 6
[junit4:junit4] 2> 200486 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[t!19 (1438269721176702976)]} 0 1
[junit4:junit4] 2> 200498 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[t!19 (1438269721185091584)]} 0 1
[junit4:junit4] 2> 200499 T607 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[t!19 (1438269721185091584)]} 0 6
[junit4:junit4] 2> 200500 T629 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[t!19]} 0 10
[junit4:junit4] 2> 200504 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[u!20 (1438269721195577344)]} 0 1
[junit4:junit4] 2> 200512 T658 C121 P41032 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[u!20 (1438269721199771648)]} 0 1
[junit4:junit4] 2> 200513 T629 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[u!20 (1438269721199771648)]} 0 6
[junit4:junit4] 2> 200518 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[v!21 (1438269721210257408)]} 0 1
[junit4:junit4] 2> 200527 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[v!21 (1438269721215500288)]} 0 1
[junit4:junit4] 2> 200529 T611 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[v!21 (1438269721215500288)]} 0 7
[junit4:junit4] 2> 200533 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[w!22 (1438269721225986048)]} 0 1
[junit4:junit4] 2> 200541 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[w!22 (1438269721230180352)]} 0 1
[junit4:junit4] 2> 200542 T611 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[w!22 (1438269721230180352)]} 0 6
[junit4:junit4] 2> 200546 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[x!23 (1438269721239617536)]} 0 1
[junit4:junit4] 2> 200559 T658 C121 P41032 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[x!23 (1438269721248006144)]} 0 2
[junit4:junit4] 2> 200560 T627 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[x!23 (1438269721248006144)]} 0 7
[junit4:junit4] 2> 200561 T611 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[x!23]} 0 11
[junit4:junit4] 2> 200565 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[y!24 (1438269721259540480)]} 0 1
[junit4:junit4] 2> 200573 T658 C121 P41032 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[y!24 (1438269721263734784)]} 0 1
[junit4:junit4] 2> 200574 T629 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[y!24 (1438269721263734784)]} 0 6
[junit4:junit4] 2> 200579 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[z!25 (1438269721274220544)]} 0 1
[junit4:junit4] 2> 200588 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[z!25 (1438269721279463424)]} 0 1
[junit4:junit4] 2> 200590 T611 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[z!25 (1438269721279463424)]} 0 7
[junit4:junit4] 2> 200594 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[a!26 (1438269721289949184)]} 0 1
[junit4:junit4] 2> 200603 T658 C121 P41032 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[a!26 (1438269721295192064)]} 0 1
[junit4:junit4] 2> 200604 T629 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[a!26 (1438269721295192064)]} 0 6
[junit4:junit4] 2> 200609 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[b!27 (1438269721305677824)]} 0 1
[junit4:junit4] 2> 200617 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[b!27 (1438269721309872128)]} 0 1
[junit4:junit4] 2> 200618 T611 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[b!27 (1438269721309872128)]} 0 6
[junit4:junit4] 2> 200622 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[c!28 (1438269721319309312)]} 0 1
[junit4:junit4] 2> 200629 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[c!28 (1438269721323503616)]} 0 1
[junit4:junit4] 2> 200630 T611 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[c!28 (1438269721323503616)]} 0 5
[junit4:junit4] 2> 200634 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[d!29 (1438269721331892224)]} 0 1
[junit4:junit4] 2> 200642 T658 C121 P41032 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[d!29 (1438269721336086528)]} 0 1
[junit4:junit4] 2> 200643 T629 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[d!29 (1438269721336086528)]} 0 6
[junit4:junit4] 2> 200647 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[e!30 (1438269721345523712)]} 0 1
[junit4:junit4] 2> 200655 T658 C121 P41032 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[e!30 (1438269721349718016)]} 0 1
[junit4:junit4] 2> 200656 T629 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[e!30 (1438269721349718016)]} 0 6
[junit4:junit4] 2> 200661 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[f!31 (1438269721359155200)]} 0 1
[junit4:junit4] 2> 200669 T658 C121 P41032 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[f!31 (1438269721364398080)]} 0 1
[junit4:junit4] 2> 200670 T629 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[f!31 (1438269721364398080)]} 0 6
[junit4:junit4] 2> 200674 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[g!32 (1438269721373835264)]} 0 1
[junit4:junit4] 2> 200686 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[g!32 (1438269721382223872)]} 0 1
[junit4:junit4] 2> 200687 T607 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[g!32 (1438269721382223872)]} 0 6
[junit4:junit4] 2> 200688 T629 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[g!32]} 0 10
[junit4:junit4] 2> 200692 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[h!33 (1438269721392709632)]} 0 1
[junit4:junit4] 2> 200700 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[h!33 (1438269721396903936)]} 0 1
[junit4:junit4] 2> 200701 T611 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[h!33 (1438269721396903936)]} 0 6
[junit4:junit4] 2> 200705 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[i!34 (1438269721406341120)]} 0 1
[junit4:junit4] 2> 200716 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[i!34 (1438269721413681152)]} 0 1
[junit4:junit4] 2> 200717 T607 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[i!34 (1438269721413681152)]} 0 6
[junit4:junit4] 2> 200718 T629 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[i!34]} 0 10
[junit4:junit4] 2> 200722 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[j!35 (1438269721424166912)]} 0 1
[junit4:junit4] 2> 200730 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[j!35 (1438269721428361216)]} 0 1
[junit4:junit4] 2> 200731 T611 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[j!35 (1438269721428361216)]} 0 6
[junit4:junit4] 2> 200736 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[k!36 (1438269721438846976)]} 0 1
[junit4:junit4] 2> 200744 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[k!36 (1438269721443041280)]} 0 1
[junit4:junit4] 2> 200745 T611 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[k!36 (1438269721443041280)]} 0 6
[junit4:junit4] 2> 200749 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[l!37 (1438269721452478464)]} 0 1
[junit4:junit4] 2> 200757 T658 C121 P41032 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[l!37 (1438269721456672768)]} 0 1
[junit4:junit4] 2> 200758 T629 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[l!37 (1438269721456672768)]} 0 6
[junit4:junit4] 2> 200762 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[m!38 (1438269721466109952)]} 0 1
[junit4:junit4] 2> 200770 T658 C121 P41032 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[m!38 (1438269721470304256)]} 0 1
[junit4:junit4] 2> 200771 T629 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[m!38 (1438269721470304256)]} 0 6
[junit4:junit4] 2> 200776 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[n!39 (1438269721480790016)]} 0 1
[junit4:junit4] 2> 200784 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[n!39 (1438269721484984320)]} 0 1
[junit4:junit4] 2> 200785 T611 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[n!39 (1438269721484984320)]} 0 6
[junit4:junit4] 2> 200790 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[o!40 (1438269721495470080)]} 0 1
[junit4:junit4] 2> 200798 T658 C121 P41032 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[o!40 (1438269721499664384)]} 0 1
[junit4:junit4] 2> 200800 T629 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[o!40 (1438269721499664384)]} 0 7
[junit4:junit4] 2> 200804 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[p!41 (1438269721510150144)]} 0 1
[junit4:junit4] 2> 200812 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[p!41 (1438269721514344448)]} 0 1
[junit4:junit4] 2> 200814 T611 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[p!41 (1438269721514344448)]} 0 7
[junit4:junit4] 2> 200818 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[q!42 (1438269721524830208)]} 0 1
[junit4:junit4] 2> 200826 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[q!42 (1438269721529024512)]} 0 1
[junit4:junit4] 2> 200827 T611 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[q!42 (1438269721529024512)]} 0 6
[junit4:junit4] 2> 200831 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[r!43 (1438269721537413120)]} 0 1
[junit4:junit4] 2> 200842 T658 C121 P41032 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[r!43 (1438269721545801728)]} 0 1
[junit4:junit4] 2> 200843 T627 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[r!43 (1438269721545801728)]} 0 6
[junit4:junit4] 2> 200844 T611 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[r!43]} 0 10
[junit4:junit4] 2> 200848 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[s!44 (1438269721556287488)]} 0 1
[junit4:junit4] 2> 200859 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[s!44 (1438269721563627520)]} 0 1
[junit4:junit4] 2> 200860 T607 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[s!44 (1438269721563627520)]} 0 6
[junit4:junit4] 2> 200861 T629 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[s!44]} 0 10
[junit4:junit4] 2> 200866 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[t!45 (1438269721575161856)]} 0 1
[junit4:junit4] 2> 200877 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[t!45 (1438269721582501888)]} 0 1
[junit4:junit4] 2> 200878 T607 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[t!45 (1438269721582501888)]} 0 6
[junit4:junit4] 2> 200879 T629 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[t!45]} 0 10
[junit4:junit4] 2> 200883 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[u!46 (1438269721592987648)]} 0 1
[junit4:junit4] 2> 200892 T658 C121 P41032 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[u!46 (1438269721598230528)]} 0 1
[junit4:junit4] 2> 200893 T629 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[u!46 (1438269721598230528)]} 0 6
[junit4:junit4] 2> 200897 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[v!47 (1438269721607667712)]} 0 1
[junit4:junit4] 2> 200905 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[v!47 (1438269721611862016)]} 0 1
[junit4:junit4] 2> 200906 T611 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[v!47 (1438269721611862016)]} 0 6
[junit4:junit4] 2> 200910 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[w!48 (1438269721620250624)]} 0 1
[junit4:junit4] 2> 200921 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[w!48 (1438269721628639232)]} 0 1
[junit4:junit4] 2> 200922 T607 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[w!48 (1438269721628639232)]} 0 6
[junit4:junit4] 2> 200923 T629 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[w!48]} 0 10
[junit4:junit4] 2> 200927 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[x!49 (1438269721639124992)]} 0 1
[junit4:junit4] 2> 200938 T658 C121 P41032 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[x!49 (1438269721646465024)]} 0 1
[junit4:junit4] 2> 200939 T627 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[x!49 (1438269721646465024)]} 0 6
[junit4:junit4] 2> 200940 T611 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[x!49]} 0 10
[junit4:junit4] 2> 200944 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[y!50 (1438269721656950784)]} 0 1
[junit4:junit4] 2> 200952 T658 C121 P41032 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[y!50 (1438269721661145088)]} 0 1
[junit4:junit4] 2> 200953 T629 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[y!50 (1438269721661145088)]} 0 6
[junit4:junit4] 2> 200957 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[z!51 (1438269721670582272)]} 0 1
[junit4:junit4] 2> 200965 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[z!51 (1438269721674776576)]} 0 1
[junit4:junit4] 2> 200966 T611 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[z!51 (1438269721674776576)]} 0 6
[junit4:junit4] 2> 200970 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[a!52 (1438269721684213760)]} 0 1
[junit4:junit4] 2> 200981 T658 C121 P41032 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[a!52 (1438269721691553792)]} 0 1
[junit4:junit4] 2> 200982 T627 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[a!52 (1438269721691553792)]} 0 6
[junit4:junit4] 2> 200983 T611 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[a!52]} 0 10
[junit4:junit4] 2> 200987 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[b!53 (1438269721702039552)]} 0 1
[junit4:junit4] 2> 200995 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[b!53 (1438269721706233856)]} 0 1
[junit4:junit4] 2> 200996 T611 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[b!53 (1438269721706233856)]} 0 6
[junit4:junit4] 2> 201000 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[c!54 (1438269721715671040)]} 0 1
[junit4:junit4] 2> 201008 T639 C125 P12539 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:62691/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[c!54 (1438269721719865344)]} 0 1
[junit4:junit4] 2> 201009 T611 C124 P62691 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[c!54 (1438269721719865344)]} 0 6
[junit4:junit4] 2> 201014 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[d!55 (1438269721730351104)]} 0 1
[junit4:junit4] 2> 201023 T658 C121 P41032 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[d!55 (1438269721734545408)]} 0 2
[junit4:junit4] 2> 201024 T629 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[d!55 (1438269721734545408)]} 0 7
[junit4:junit4] 2> 201028 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[e!56 (1438269721745031168)]} 0 1
[junit4:junit4] 2> 201036 T658 C121 P41032 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={distrib.from=http://127.0.0.1:57938/l_qkp/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[e!56 (1438269721749225472)]} 0 1
[junit4:junit4] 2> 201037 T629 C122 P57938 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[e!56 (1438269721749225472)]} 0 6
[junit4:junit4] 2> 201041 T587 C123 P49264 oasup.LogUpdateProcessor.finish [collection1] webapp=/l_qkp/r path=/update params={wt=javabin&version=2} {add=[f!57 (1438269721758662656)]} 0 1
[junit4:junit4] 2> 201049 T658 C121 P41032 oasup.LogUpdateProcessor.finish [collection1] we
[...truncated too long message...]
t4:junit4] 1> "base_url":"http://127.0.0.1:61912/did/j"}}}},
[junit4:junit4] 1> "router":"compositeId"},
[junit4:junit4] 1> "control_collection":{
[junit4:junit4] 1> "shards":{"shard1":{
[junit4:junit4] 1> "range":"80000000-7fffffff",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "replicas":{"127.0.0.1:33192_did%2Fj_collection1":{
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"control_collection",
[junit4:junit4] 1> "node_name":"127.0.0.1:33192_did%2Fj",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:33192/did/j",
[junit4:junit4] 1> "leader":"true"}}}},
[junit4:junit4] 1> "router":"compositeId"}}
[junit4:junit4] 1> /solr/aliases.json (0)
[junit4:junit4] 1> /solr/live_nodes (4)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:61912_did%2Fj (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:22110_did%2Fj (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:45840_did%2Fj (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:33192_did%2Fj (0)
[junit4:junit4] 1> /solr/overseer (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1> /solr/overseer/queue (0)
[junit4:junit4] 1> /solr/overseer/queue-work (0)
[junit4:junit4] 1> /solr/overseer/collection-queue-work (0)
[junit4:junit4] 1> /solr/collections (2)
[junit4:junit4] 1> /solr/collections/collection1 (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/collection1/shards (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election (3)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89891911911931914-127.0.0.1:22110_did%2Fj_collection1-n_0000000002 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89891911911931916-127.0.0.1:61912_did%2Fj_collection1-n_0000000003 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89891911911931918-127.0.0.1:45840_did%2Fj_collection1-n_0000000004 (0)
[junit4:junit4] 1> /solr/collections/collection1/leaders (1)
[junit4:junit4] 1> /solr/collections/collection1/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:22110_did%2Fj",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:22110/did/j"}
[junit4:junit4] 1> /solr/collections/control_collection (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/control_collection/shards (0)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election/89891911911931907-127.0.0.1:33192_did%2Fj_collection1-n_0000000000 (0)
[junit4:junit4] 1> /solr/collections/control_collection/leaders (1)
[junit4:junit4] 1> /solr/collections/control_collection/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:33192_did%2Fj",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:33192/did/j"}
[junit4:junit4] 1> /solr/overseer_elect (2)
[junit4:junit4] 1> /solr/overseer_elect/election (4)
[junit4:junit4] 1> /solr/overseer_elect/election/89891911911931918-127.0.0.1:45840_did%2Fj-n_0000000005 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89891911911931916-127.0.0.1:61912_did%2Fj-n_0000000004 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89891911911931914-127.0.0.1:22110_did%2Fj-n_0000000003 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89891911911931907-127.0.0.1:33192_did%2Fj-n_0000000000 (0)
[junit4:junit4] 1> /solr/overseer_elect/leader (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"id":"89891911911931907-127.0.0.1:33192_did%2Fj-n_0000000000"}
[junit4:junit4] 1> /zookeeper (1)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SyncSliceTest -Dtests.method=testDistribSearch -Dtests.seed=8982A827897334FE -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=en_GB -Dtests.timezone=Kwajalein -Dtests.file.encoding=US-ASCII
[junit4:junit4] FAILURE 515s J0 | SyncSliceTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:45840/did/j/collection1lastClient and got 255 from http://127.0.0.1:22110/did/j/collection1
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([8982A827897334FE:864263FFE2C54C2]:0)
[junit4:junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
[junit4:junit4] > at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> 1544999 T2540 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 515395 T2539 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 2 leaked thread(s).
[junit4:junit4] 2> 1546025 T2634 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 1546025 T2634 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1546025 T2634 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> NOTE: test params are: codec=Lucene41, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=en_GB, timezone=Kwajalein
[junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=2,free=250698032,total=399507456
[junit4:junit4] 2> NOTE: All tests run in this JVM: [FieldMutatingUpdateProcessorTest, CacheHeaderTest, LegacyHTMLStripCharFilterTest, TestExtendedDismaxParser, BinaryUpdateRequestHandlerTest, ReturnFieldsTest, TestPHPSerializedResponseWriter, TestFuzzyAnalyzedSuggestions, PolyFieldTest, TestTrie, TestLFUCache, TestDocumentBuilder, PreAnalyzedFieldTest, TestPseudoReturnFields, CircularListTest, PreAnalyzedUpdateProcessorTest, TestPluginEnable, ScriptEngineTest, SynonymTokenizerTest, SolrIndexSplitterTest, TermVectorComponentTest, DistributedTermsComponentTest, ZkCLITest, TestCloudManagedSchemaAddField, TestSolrCoreProperties, TestBadConfig, TestSolrXmlPersistence, ShardSplitTest, DocValuesTest, UUIDFieldTest, TestFastWriter, AlternateDirectoryTest, CachingDirectoryFactoryTest, TestQuerySenderListener, TestSolrQueryParser, TestLMDirichletSimilarityFactory, TestDocSet, ChaosMonkeyNothingIsSafeTest, TestSolrIndexConfig, FileBasedSpellCheckerTest, TestCharFilters, SolrCoreTest, SpellingQueryConverterTest, LoggingHandlerTest, TestSearchPerf, TestSchemaSimilarityResource, BadComponentTest, ZkControllerTest, PluginInfoTest, MBeansHandlerTest, AnalysisAfterCoreReloadTest, PeerSyncTest, TestPhraseSuggestions, StatelessScriptUpdateProcessorFactoryTest, SliceStateUpdateTest, SystemInfoHandlerTest, TestStressVersions, RegexBoostProcessorTest, AliasIntegrationTest, OverseerTest, BasicFunctionalityTest, TestLRUCache, UpdateRequestProcessorFactoryTest, TestCoreContainer, TestGroupingSearch, BadCopyFieldTest, TestFunctionQuery, TestPerFieldSimilarity, TestSolrDeletionPolicy1, TestFieldCollectionResource, BasicDistributedZkTest, BasicDistributedZk2Test, SyncSliceTest]
[junit4:junit4] Completed on J0 in 516.69s, 1 test, 1 failure <<< FAILURES!
[...truncated 438 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:386: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:366: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:181: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:437: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1243: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:887: There were test failures: 297 suites, 1252 tests, 1 error, 2 failures, 13 ignored (7 assumptions)
Total time: 57 minutes 24 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4075 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4075/
1 tests failed.
FAILED: org.apache.solr.cloud.ChaosMonkeyShardSplitTest.testDistribSearch
Error Message:
Server at http://127.0.0.1:51389/_zgt/j returned non ok status:500, message:Server Error
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:51389/_zgt/j returned non ok status:500, message:Server Error
at __randomizedtesting.SeedInfo.seed([EC51938F7B770E09:6DB71D970C286E35]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
at org.apache.solr.cloud.ChaosMonkeyShardSplitTest.doTest(ChaosMonkeyShardSplitTest.java:136)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 10025 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ChaosMonkeyShardSplitTest
[junit4:junit4] 2> 1432053 T4163 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /_zgt/j
[junit4:junit4] 2> 1432058 T4163 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./solrtest-ChaosMonkeyShardSplitTest-1371591288177
[junit4:junit4] 2> 1432060 T4163 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 1432061 T4164 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 1432162 T4163 oasc.ZkTestServer.run start zk server on port:56759
[junit4:junit4] 2> 1432163 T4163 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1432167 T4170 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@119bbed name:ZooKeeperConnection Watcher:127.0.0.1:56759 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1432168 T4163 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1432168 T4163 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 1432178 T4163 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1432190 T4172 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@788163b3 name:ZooKeeperConnection Watcher:127.0.0.1:56759/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1432191 T4163 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1432191 T4163 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 1432200 T4163 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 1432209 T4163 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 1432212 T4163 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 1432215 T4163 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 1432216 T4163 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 1432227 T4163 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 1432228 T4163 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 1432331 T4163 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 1432332 T4163 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 1432335 T4163 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 1432336 T4163 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 1432340 T4163 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 1432340 T4163 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 1432344 T4163 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 1432345 T4163 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 1432348 T4163 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 1432349 T4163 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 1432352 T4163 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 1432353 T4163 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 1432356 T4163 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 1432357 T4163 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 1432670 T4163 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1432678 T4163 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:39731
[junit4:junit4] 2> 1432679 T4163 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1432679 T4163 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1432680 T4163 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371591288478
[junit4:junit4] 2> 1432680 T4163 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371591288478/solr.xml
[junit4:junit4] 2> 1432681 T4163 oasc.CoreContainer.<init> New CoreContainer 1817396321
[junit4:junit4] 2> 1432682 T4163 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371591288478/'
[junit4:junit4] 2> 1432682 T4163 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371591288478/'
[junit4:junit4] 2> 1432815 T4163 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1432816 T4163 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1432817 T4163 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1432817 T4163 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1432818 T4163 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1432818 T4163 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1432819 T4163 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1432820 T4163 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1432820 T4163 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1432821 T4163 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1432838 T4163 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1432840 T4163 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56759/solr
[junit4:junit4] 2> 1432840 T4163 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1432841 T4163 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1432850 T4183 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@38a32fbb name:ZooKeeperConnection Watcher:127.0.0.1:56759 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1432851 T4163 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1432854 T4163 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1432867 T4163 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1432869 T4185 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@38b56235 name:ZooKeeperConnection Watcher:127.0.0.1:56759/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1432870 T4163 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1432872 T4163 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 1432881 T4163 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 1432884 T4163 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 1432891 T4163 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39731__zgt%2Fj
[junit4:junit4] 2> 1432893 T4163 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:39731__zgt%2Fj
[junit4:junit4] 2> 1432896 T4163 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 1432907 T4163 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 1432917 T4163 oasc.Overseer.start Overseer (id=89888606662426627-127.0.0.1:39731__zgt%2Fj-n_0000000000) starting
[junit4:junit4] 2> 1432921 T4163 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 1432924 T4187 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 1432925 T4163 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 1432927 T4163 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 1432929 T4163 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1432932 T4186 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 1432936 T4188 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1432937 T4188 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1434435 T4186 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1434436 T4186 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:39731__zgt%2Fj",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:39731/_zgt/j"}
[junit4:junit4] 2> 1434437 T4186 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 1434437 T4186 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1434449 T4185 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1434940 T4188 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371591288478/collection1
[junit4:junit4] 2> 1434940 T4188 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 1434941 T4188 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1434941 T4188 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 1434943 T4188 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371591288478/collection1/'
[junit4:junit4] 2> 1434945 T4188 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371591288478/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1434945 T4188 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371591288478/collection1/lib/README' to classloader
[junit4:junit4] 2> 1435002 T4188 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1435067 T4188 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1435168 T4188 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1435178 T4188 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1435847 T4188 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1435852 T4188 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1435855 T4188 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1435860 T4188 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1435884 T4188 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1435884 T4188 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371591288478/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/control/data/
[junit4:junit4] 2> 1435885 T4188 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7e6cb446
[junit4:junit4] 2> 1435885 T4188 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1435886 T4188 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/control/data
[junit4:junit4] 2> 1435886 T4188 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/control/data/index/
[junit4:junit4] 2> 1435886 T4188 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1435887 T4188 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/control/data/index
[junit4:junit4] 2> 1435892 T4188 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@30f848c6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3de53383),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1435892 T4188 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1435895 T4188 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1435895 T4188 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1435896 T4188 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1435896 T4188 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1435897 T4188 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1435897 T4188 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1435897 T4188 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1435898 T4188 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1435898 T4188 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1435913 T4188 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1435920 T4188 oass.SolrIndexSearcher.<init> Opening Searcher@6f524b02 main
[junit4:junit4] 2> 1435921 T4188 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/control/data/tlog
[junit4:junit4] 2> 1435922 T4188 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1435922 T4188 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1435926 T4189 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6f524b02 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1435928 T4188 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1435928 T4188 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:39731/_zgt/j collection:control_collection shard:shard1
[junit4:junit4] 2> 1435929 T4188 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 1435936 T4188 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1435938 T4188 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1435939 T4188 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1435939 T4188 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:39731/_zgt/j/collection1/
[junit4:junit4] 2> 1435939 T4188 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1435940 T4188 oasc.SyncStrategy.syncToMe http://127.0.0.1:39731/_zgt/j/collection1/ has no replicas
[junit4:junit4] 2> 1435940 T4188 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:39731/_zgt/j/collection1/
[junit4:junit4] 2> 1435940 T4188 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 1435954 T4186 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1435961 T4185 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1435997 T4188 oasc.ZkController.register We are http://127.0.0.1:39731/_zgt/j/collection1/ and leader is http://127.0.0.1:39731/_zgt/j/collection1/
[junit4:junit4] 2> 1435997 T4188 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:39731/_zgt/j
[junit4:junit4] 2> 1435997 T4188 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1435997 T4188 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1435998 T4188 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1435999 T4188 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1436000 T4163 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1436001 T4163 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1436002 T4163 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1436022 T4163 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 1436024 T4163 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1436026 T4192 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a824122 name:ZooKeeperConnection Watcher:127.0.0.1:56759/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1436026 T4163 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1436028 T4163 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1436031 T4163 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 1436357 T4163 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1436360 T4163 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:51389
[junit4:junit4] 2> 1436361 T4163 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1436362 T4163 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1436362 T4163 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371591292148
[junit4:junit4] 2> 1436363 T4163 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371591292148/solr.xml
[junit4:junit4] 2> 1436363 T4163 oasc.CoreContainer.<init> New CoreContainer 1323136020
[junit4:junit4] 2> 1436364 T4163 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371591292148/'
[junit4:junit4] 2> 1436365 T4163 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371591292148/'
[junit4:junit4] 2> 1436513 T4163 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1436514 T4163 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1436515 T4163 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1436515 T4163 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1436516 T4163 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1436516 T4163 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1436517 T4163 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1436517 T4163 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1436518 T4163 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1436519 T4163 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1436536 T4163 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1436537 T4163 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56759/solr
[junit4:junit4] 2> 1436537 T4163 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1436538 T4163 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1436541 T4203 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3f9b217c name:ZooKeeperConnection Watcher:127.0.0.1:56759 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1436543 T4163 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1436545 T4163 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1436557 T4163 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1436559 T4205 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6817bf78 name:ZooKeeperConnection Watcher:127.0.0.1:56759/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1436559 T4163 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1436571 T4163 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1437466 T4186 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1437467 T4186 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:39731__zgt%2Fj",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:39731/_zgt/j"}
[junit4:junit4] 2> 1437472 T4185 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1437472 T4192 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1437472 T4205 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1437574 T4163 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51389__zgt%2Fj
[junit4:junit4] 2> 1437576 T4163 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51389__zgt%2Fj
[junit4:junit4] 2> 1437578 T4205 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1437578 T4185 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1437579 T4192 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1437579 T4192 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1437580 T4205 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1437580 T4185 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1437593 T4206 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1437593 T4206 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1438978 T4186 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1438979 T4186 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51389__zgt%2Fj",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51389/_zgt/j"}
[junit4:junit4] 2> 1438980 T4186 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 1438980 T4186 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1438991 T4185 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1438991 T4192 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1438991 T4205 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1439597 T4206 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371591292148/collection1
[junit4:junit4] 2> 1439597 T4206 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1439599 T4206 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1439599 T4206 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1439602 T4206 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371591292148/collection1/'
[junit4:junit4] 2> 1439605 T4206 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371591292148/collection1/lib/README' to classloader
[junit4:junit4] 2> 1439606 T4206 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371591292148/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1439666 T4206 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1439732 T4206 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1439833 T4206 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1439842 T4206 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1440472 T4206 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1440477 T4206 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1440480 T4206 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1440485 T4206 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1440509 T4206 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1440509 T4206 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371591292148/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty1/
[junit4:junit4] 2> 1440510 T4206 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7e6cb446
[junit4:junit4] 2> 1440510 T4206 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1440511 T4206 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty1
[junit4:junit4] 2> 1440511 T4206 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty1/index/
[junit4:junit4] 2> 1440512 T4206 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1440513 T4206 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty1/index
[junit4:junit4] 2> 1440517 T4206 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@42817fe5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@60a6d3fc),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1440518 T4206 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1440521 T4206 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1440522 T4206 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1440522 T4206 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1440523 T4206 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1440524 T4206 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1440524 T4206 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1440524 T4206 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1440525 T4206 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1440525 T4206 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1440542 T4206 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1440550 T4206 oass.SolrIndexSearcher.<init> Opening Searcher@5d6e49e5 main
[junit4:junit4] 2> 1440550 T4206 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty1/tlog
[junit4:junit4] 2> 1440551 T4206 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1440552 T4206 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1440556 T4207 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5d6e49e5 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1440558 T4206 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1440559 T4206 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:51389/_zgt/j collection:collection1 shard:shard1
[junit4:junit4] 2> 1440559 T4206 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 1440567 T4206 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1440570 T4206 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1440570 T4206 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1440570 T4206 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:51389/_zgt/j/collection1/
[junit4:junit4] 2> 1440570 T4206 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1440571 T4206 oasc.SyncStrategy.syncToMe http://127.0.0.1:51389/_zgt/j/collection1/ has no replicas
[junit4:junit4] 2> 1440571 T4206 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:51389/_zgt/j/collection1/
[junit4:junit4] 2> 1440571 T4206 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 1441997 T4186 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1442005 T4185 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1442005 T4205 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1442005 T4192 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1442056 T4206 oasc.ZkController.register We are http://127.0.0.1:51389/_zgt/j/collection1/ and leader is http://127.0.0.1:51389/_zgt/j/collection1/
[junit4:junit4] 2> 1442056 T4206 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51389/_zgt/j
[junit4:junit4] 2> 1442056 T4206 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1442056 T4206 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1442057 T4206 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1442059 T4206 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1442061 T4163 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1442061 T4163 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1442062 T4163 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1442349 T4163 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1442353 T4163 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:51395
[junit4:junit4] 2> 1442354 T4163 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1442354 T4163 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1442355 T4163 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371591298195
[junit4:junit4] 2> 1442355 T4163 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371591298195/solr.xml
[junit4:junit4] 2> 1442356 T4163 oasc.CoreContainer.<init> New CoreContainer 1628569058
[junit4:junit4] 2> 1442357 T4163 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371591298195/'
[junit4:junit4] 2> 1442358 T4163 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371591298195/'
[junit4:junit4] 2> 1442493 T4163 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1442494 T4163 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1442495 T4163 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1442495 T4163 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1442496 T4163 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1442496 T4163 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1442497 T4163 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1442497 T4163 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1442498 T4163 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1442499 T4163 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1442514 T4163 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1442515 T4163 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56759/solr
[junit4:junit4] 2> 1442515 T4163 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1442516 T4163 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1442587 T4219 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5fb5023f name:ZooKeeperConnection Watcher:127.0.0.1:56759 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1442587 T4163 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1442591 T4163 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1442604 T4163 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1442614 T4221 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@479025b2 name:ZooKeeperConnection Watcher:127.0.0.1:56759/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1442614 T4163 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1442625 T4163 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1443515 T4186 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1443517 T4186 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51389__zgt%2Fj",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51389/_zgt/j"}
[junit4:junit4] 2> 1443617 T4185 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1443618 T4192 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1443618 T4205 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1443618 T4221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1443629 T4163 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51395__zgt%2Fj
[junit4:junit4] 2> 1443649 T4163 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51395__zgt%2Fj
[junit4:junit4] 2> 1443695 T4185 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1443695 T4205 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1443696 T4192 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1443696 T4221 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1443696 T4192 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1443696 T4221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1443697 T4185 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1443697 T4205 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1443754 T4222 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1443754 T4222 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1445122 T4186 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1445124 T4186 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51395__zgt%2Fj",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51395/_zgt/j"}
[junit4:junit4] 2> 1445124 T4186 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 1445124 T4186 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 1445151 T4185 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1445151 T4221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1445151 T4205 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1445151 T4192 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1445799 T4222 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371591298195/collection1
[junit4:junit4] 2> 1445799 T4222 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1445800 T4222 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1445800 T4222 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1445802 T4222 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371591298195/collection1/'
[junit4:junit4] 2> 1445804 T4222 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371591298195/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1445805 T4222 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371591298195/collection1/lib/README' to classloader
[junit4:junit4] 2> 1445865 T4222 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1445941 T4222 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1446043 T4222 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1446051 T4222 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1446647 T4222 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1446651 T4222 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1446654 T4222 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1446659 T4222 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1446683 T4222 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1446684 T4222 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371591298195/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty2/
[junit4:junit4] 2> 1446685 T4222 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7e6cb446
[junit4:junit4] 2> 1446685 T4222 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1446686 T4222 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty2
[junit4:junit4] 2> 1446686 T4222 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty2/index/
[junit4:junit4] 2> 1446687 T4222 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1446688 T4222 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty2/index
[junit4:junit4] 2> 1446692 T4222 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4006bc76 lockFactory=org.apache.lucene.store.NativeFSLockFactory@750ff575),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1446692 T4222 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1446696 T4222 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1446696 T4222 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1446697 T4222 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1446697 T4222 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1446698 T4222 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1446698 T4222 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1446699 T4222 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1446699 T4222 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1446700 T4222 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1446718 T4222 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1446726 T4222 oass.SolrIndexSearcher.<init> Opening Searcher@2baf14a6 main
[junit4:junit4] 2> 1446726 T4222 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty2/tlog
[junit4:junit4] 2> 1446727 T4222 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1446728 T4222 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1446733 T4223 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2baf14a6 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1446735 T4222 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1446736 T4222 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:51395/_zgt/j collection:collection1 shard:shard2
[junit4:junit4] 2> 1446736 T4222 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 1446759 T4222 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1446762 T4222 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1446762 T4222 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1446763 T4222 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:51395/_zgt/j/collection1/
[junit4:junit4] 2> 1446763 T4222 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1446763 T4222 oasc.SyncStrategy.syncToMe http://127.0.0.1:51395/_zgt/j/collection1/ has no replicas
[junit4:junit4] 2> 1446763 T4222 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:51395/_zgt/j/collection1/
[junit4:junit4] 2> 1446764 T4222 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 1448157 T4186 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1448190 T4192 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1448190 T4221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1448190 T4205 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1448190 T4185 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1448205 T4222 oasc.ZkController.register We are http://127.0.0.1:51395/_zgt/j/collection1/ and leader is http://127.0.0.1:51395/_zgt/j/collection1/
[junit4:junit4] 2> 1448205 T4222 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51395/_zgt/j
[junit4:junit4] 2> 1448205 T4222 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1448205 T4222 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1448206 T4222 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1448210 T4222 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1448212 T4163 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1448212 T4163 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1448213 T4163 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1448483 T4163 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1448486 T4163 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:51404
[junit4:junit4] 2> 1448487 T4163 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1448488 T4163 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1448488 T4163 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371591304346
[junit4:junit4] 2> 1448489 T4163 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371591304346/solr.xml
[junit4:junit4] 2> 1448489 T4163 oasc.CoreContainer.<init> New CoreContainer 1114129849
[junit4:junit4] 2> 1448490 T4163 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371591304346/'
[junit4:junit4] 2> 1448491 T4163 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371591304346/'
[junit4:junit4] 2> 1448647 T4163 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1448648 T4163 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1448648 T4163 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1448649 T4163 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1448649 T4163 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1448650 T4163 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1448650 T4163 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1448651 T4163 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1448651 T4163 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1448652 T4163 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1448668 T4163 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1448668 T4163 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56759/solr
[junit4:junit4] 2> 1448669 T4163 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1448670 T4163 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1448714 T4235 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7f3b4a16 name:ZooKeeperConnection Watcher:127.0.0.1:56759 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1448714 T4163 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1448718 T4163 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1448729 T4163 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1448774 T4237 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2f59ace6 name:ZooKeeperConnection Watcher:127.0.0.1:56759/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1448775 T4163 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1448813 T4163 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1449696 T4186 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1449697 T4186 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51395__zgt%2Fj",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51395/_zgt/j"}
[junit4:junit4] 2> 1449723 T4192 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1449723 T4221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1449723 T4205 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1449723 T4185 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1449723 T4237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1449817 T4163 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51404__zgt%2Fj
[junit4:junit4] 2> 1449819 T4163 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51404__zgt%2Fj
[junit4:junit4] 2> 1449821 T4237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1449821 T4205 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1449822 T4192 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1449821 T4185 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1449822 T4192 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1449822 T4221 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1449823 T4221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1449823 T4237 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1449823 T4205 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1449832 T4185 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1449838 T4238 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1449838 T4238 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1451236 T4186 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1451237 T4186 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51404__zgt%2Fj",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51404/_zgt/j"}
[junit4:junit4] 2> 1451237 T4186 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 1451237 T4186 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1451251 T4192 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1451251 T4221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1451251 T4205 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1451251 T4237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1451251 T4185 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1451842 T4238 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371591304346/collection1
[junit4:junit4] 2> 1451842 T4238 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1451843 T4238 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1451843 T4238 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1451845 T4238 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371591304346/collection1/'
[junit4:junit4] 2> 1451846 T4238 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371591304346/collection1/lib/README' to classloader
[junit4:junit4] 2> 1451847 T4238 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371591304346/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1451900 T4238 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1451962 T4238 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1452063 T4238 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1452071 T4238 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1452641 T4238 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1452645 T4238 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1452647 T4238 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1452653 T4238 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1452676 T4238 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1452677 T4238 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371591304346/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty3/
[junit4:junit4] 2> 1452677 T4238 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7e6cb446
[junit4:junit4] 2> 1452678 T4238 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1452678 T4238 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty3
[junit4:junit4] 2> 1452679 T4238 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty3/index/
[junit4:junit4] 2> 1452679 T4238 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1452680 T4238 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty3/index
[junit4:junit4] 2> 1452684 T4238 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@70fae8b2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@45a324b1),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1452684 T4238 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1452687 T4238 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1452688 T4238 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1452688 T4238 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1452689 T4238 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1452690 T4238 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1452690 T4238 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1452690 T4238 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1452691 T4238 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1452691 T4238 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1452708 T4238 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1452715 T4238 oass.SolrIndexSearcher.<init> Opening Searcher@48f86206 main
[junit4:junit4] 2> 1452716 T4238 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty3/tlog
[junit4:junit4] 2> 1452717 T4238 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1452718 T4238 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1452722 T4239 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@48f86206 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1452725 T4238 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1452725 T4238 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:51404/_zgt/j collection:collection1 shard:shard1
[junit4:junit4] 2> 1452729 T4238 oasc.ZkController.register We are http://127.0.0.1:51404/_zgt/j/collection1/ and leader is http://127.0.0.1:51389/_zgt/j/collection1/
[junit4:junit4] 2> 1452729 T4238 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51404/_zgt/j
[junit4:junit4] 2> 1452730 T4238 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 1452730 T4238 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C2148 name=collection1 org.apache.solr.core.SolrCore@632eae13 url=http://127.0.0.1:51404/_zgt/j/collection1 node=127.0.0.1:51404__zgt%2Fj C2148_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:51404__zgt%2Fj, base_url=http://127.0.0.1:51404/_zgt/j}
[junit4:junit4] 2> 1452730 T4240 C2148 P51404 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 1452731 T4238 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1452731 T4240 C2148 P51404 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 1452731 T4240 C2148 P51404 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 1452732 T4240 C2148 P51404 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1452732 T4163 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1452733 T4163 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1452733 T4240 C2148 P51404 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1452734 T4163 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1452760 T4195 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 1452764 T4186 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1452765 T4186 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51404__zgt%2Fj",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51404/_zgt/j"}
[junit4:junit4] 2> 1452778 T4192 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1452778 T4221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1452778 T4205 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1452778 T4237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1452778 T4185 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1453011 T4163 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1453014 T4163 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:51415
[junit4:junit4] 2> 1453014 T4163 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1453015 T4163 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1453016 T4163 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371591308876
[junit4:junit4] 2> 1453016 T4163 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371591308876/solr.xml
[junit4:junit4] 2> 1453017 T4163 oasc.CoreContainer.<init> New CoreContainer 241702131
[junit4:junit4] 2> 1453017 T4163 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371591308876/'
[junit4:junit4] 2> 1453018 T4163 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371591308876/'
[junit4:junit4] 2> 1453146 T4163 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1453147 T4163 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1453148 T4163 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1453148 T4163 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1453149 T4163 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1453149 T4163 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1453150 T4163 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1453150 T4163 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1453151 T4163 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1453151 T4163 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1453169 T4163 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1453170 T4163 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56759/solr
[junit4:junit4] 2> 1453170 T4163 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1453171 T4163 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1453174 T4252 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4497e062 name:ZooKeeperConnection Watcher:127.0.0.1:56759 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1453175 T4163 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1453191 T4163 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1453203 T4163 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1453205 T4254 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@16f3fb3 name:ZooKeeperConnection Watcher:127.0.0.1:56759/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1453206 T4163 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1453217 T4163 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1453761 T4195 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 1453761 T4195 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=4&state=recovering&nodeName=127.0.0.1:51404__zgt%252Fj&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 1454220 T4163 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51415__zgt%2Fj
[junit4:junit4] 2> 1454223 T4163 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51415__zgt%2Fj
[junit4:junit4] 2> 1454227 T4185 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1454227 T4237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1454227 T4192 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1454227 T4205 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1454228 T4192 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1454227 T4221 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1454227 T4254 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1454228 T4221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1454229 T4185 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1454230 T4237 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1454230 T4205 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1454236 T4255 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1454236 T4255 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1454289 T4186 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1454290 T4186 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51415__zgt%2Fj",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51415/_zgt/j"}
[junit4:junit4] 2> 1454291 T4186 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 1454291 T4186 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 1454296 T4254 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1454296 T4237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1454296 T4221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1454296 T4205 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1454296 T4185 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1454296 T4192 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1455246 T4255 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371591308876/collection1
[junit4:junit4] 2> 1455246 T4255 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1455247 T4255 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1455247 T4255 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1455249 T4255 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371591308876/collection1/'
[junit4:junit4] 2> 1455251 T4255 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371591308876/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1455251 T4255 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371591308876/collection1/lib/README' to classloader
[junit4:junit4] 2> 1455304 T4255 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1455367 T4255 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1455468 T4255 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1455475 T4255 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> ASYNC NEW_CORE C2149 name=collection1 org.apache.solr.core.SolrCore@632eae13 url=http://127.0.0.1:51404/_zgt/j/collection1 node=127.0.0.1:51404__zgt%2Fj C2149_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:51404__zgt%2Fj, base_url=http://127.0.0.1:51404/_zgt/j}
[junit4:junit4] 2> 1455763 T4240 C2149 P51404 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:51389/_zgt/j/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 1455763 T4240 C2149 P51404 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:51404/_zgt/j START replicas=[http://127.0.0.1:51389/_zgt/j/collection1/] nUpdates=100
[junit4:junit4] 2> 1455764 T4240 C2149 P51404 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 1455765 T4240 C2149 P51404 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 1455765 T4240 C2149 P51404 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 1455765 T4240 C2149 P51404 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 1455766 T4240 C2149 P51404 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 1455766 T4240 C2149 P51404 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:51389/_zgt/j/collection1/. core=collection1
[junit4:junit4] 2> 1455766 T4240 C2149 P51404 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C2150 name=collection1 org.apache.solr.core.SolrCore@70ac15f7 url=http://127.0.0.1:51389/_zgt/j/collection1 node=127.0.0.1:51389__zgt%2Fj C2150_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:51389__zgt%2Fj, base_url=http://127.0.0.1:51389/_zgt/j, leader=true}
[junit4:junit4] 2> 1455776 T4196 C2150 P51389 oasc.SolrCore.execute [collection1] webapp=/_zgt/j path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 1455779 T4197 C2150 P51389 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1455782 T4197 C2150 P51389 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@42817fe5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@60a6d3fc),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1455783 T4197 C2150 P51389 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1455783 T4197 C2150 P51389 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@42817fe5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@60a6d3fc),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@42817fe5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@60a6d3fc),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1455784 T4197 C2150 P51389 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1455785 T4197 C2150 P51389 oass.SolrIndexSearcher.<init> Opening Searcher@9f5155c realtime
[junit4:junit4] 2> 1455785 T4197 C2150 P51389 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1455785 T4197 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 1455786 T4240 C2149 P51404 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1455787 T4240 C2149 P51404 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 1455789 T4198 C2150 P51389 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1455790 T4198 C2150 P51389 oasc.SolrCore.execute [collection1] webapp=/_zgt/j path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2
[junit4:junit4] 2> 1455790 T4240 C2149 P51404 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 1455790 T4240 C2149 P51404 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 1455791 T4240 C2149 P51404 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 1455793 T4198 C2150 P51389 oasc.SolrCore.execute [collection1] webapp=/_zgt/j path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 1455794 T4240 C2149 P51404 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 1455795 T4240 C2149 P51404 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty3/index.20130618163511911
[junit4:junit4] 2> 1455795 T4240 C2149 P51404 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@63ece2cb lockFactory=org.apache.lucene.store.NativeFSLockFactory@66caafc7) fullCopy=false
[junit4:junit4] 2> 1455798 T4198 C2150 P51389 oasc.SolrCore.execute [collection1] webapp=/_zgt/j path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 1455799 T4240 C2149 P51404 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 1455800 T4240 C2149 P51404 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 1455801 T4240 C2149 P51404 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 1455803 T4240 C2149 P51404 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@70fae8b2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@45a324b1),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@70fae8b2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@45a324b1),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1455803 T4240 C2149 P51404 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1455804 T4240 C2149 P51404 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 1455804 T4240 C2149 P51404 oass.SolrIndexSearcher.<init> Opening Searcher@8e987a5 main
[junit4:junit4] 2> 1455805 T4239 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@8e987a5 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1455806 T4240 C2149 P51404 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty3/index.20130618163511911 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty3/index.20130618163511911;done=true>>]
[junit4:junit4] 2> 1455806 T4240 C2149 P51404 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty3/index.20130618163511911
[junit4:junit4] 2> 1455806 T4240 C2149 P51404 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty3/index.20130618163511911
[junit4:junit4] 2> 1455807 T4240 C2149 P51404 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 1455807 T4240 C2149 P51404 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 1455807 T4240 C2149 P51404 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1455807 T4240 C2149 P51404 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1455809 T4240 C2149 P51404 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 1456090 T4255 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1456095 T4255 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1456097 T4255 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1456103 T4255 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1456126 T4255 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1456126 T4255 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371591308876/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty4/
[junit4:junit4] 2> 1456126 T4255 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7e6cb446
[junit4:junit4] 2> 1456127 T4255 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1456128 T4255 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty4
[junit4:junit4] 2> 1456128 T4255 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty4/index/
[junit4:junit4] 2> 1456128 T4255 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1456129 T4255 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty4/index
[junit4:junit4] 2> 1456133 T4255 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7564e83e lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b98a787),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1456133 T4255 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1456136 T4255 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1456137 T4255 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1456137 T4255 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1456138 T4255 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1456139 T4255 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1456139 T4255 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1456140 T4255 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1456141 T4255 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1456141 T4255 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1456158 T4255 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1456167 T4255 oass.SolrIndexSearcher.<init> Opening Searcher@19855a5d main
[junit4:junit4] 2> 1456167 T4255 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty4/tlog
[junit4:junit4] 2> 1456168 T4255 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1456168 T4255 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1456173 T4258 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@19855a5d main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1456175 T4255 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1456175 T4255 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:51415/_zgt/j collection:collection1 shard:shard2
[junit4:junit4] 2> 1456179 T4255 oasc.ZkController.register We are http://127.0.0.1:51415/_zgt/j/collection1/ and leader is http://127.0.0.1:51395/_zgt/j/collection1/
[junit4:junit4] 2> 1456179 T4255 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51415/_zgt/j
[junit4:junit4] 2> 1456179 T4255 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 1456180 T4255 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C2151 name=collection1 org.apache.solr.core.SolrCore@52c5c681 url=http://127.0.0.1:51415/_zgt/j/collection1 node=127.0.0.1:51415__zgt%2Fj C2151_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:51415__zgt%2Fj, base_url=http://127.0.0.1:51415/_zgt/j}
[junit4:junit4] 2> 1456180 T4259 C2151 P51415 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 1456181 T4259 C2151 P51415 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 1456181 T4255 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1456181 T4259 C2151 P51415 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 1456181 T4259 C2151 P51415 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1456182 T4163 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1456182 T4163 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1456182 T4259 C2151 P51415 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1456183 T4163 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1456195 T4163 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1456196 T4211 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 1456196 T4163 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 1456197 T4163 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 1456197 T4163 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1457199 T4163 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1457309 T4186 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1457310 T4186 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51404__zgt%2Fj",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51404/_zgt/j"}
[junit4:junit4] 2> 1457314 T4186 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51415__zgt%2Fj",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51415/_zgt/j"}
[junit4:junit4] 2> 1457329 T4254 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1457329 T4221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1457329 T4205 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1457329 T4237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1457329 T4185 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1457329 T4192 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1458198 T4211 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 1458198 T4211 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=5&state=recovering&nodeName=127.0.0.1:51415__zgt%252Fj&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2002
[junit4:junit4] 2> 1458201 T4163 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1459203 T4163 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C2151_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:51415__zgt%2Fj, base_url=http://127.0.0.1:51415/_zgt/j}
[junit4:junit4] 2> 1460200 T4259 C2151 P51415 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:51395/_zgt/j/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 1460200 T4259 C2151 P51415 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:51415/_zgt/j START replicas=[http://127.0.0.1:51395/_zgt/j/collection1/] nUpdates=100
[junit4:junit4] 2> 1460201 T4259 C2151 P51415 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 1460202 T4259 C2151 P51415 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 1460202 T4259 C2151 P51415 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 1460202 T4259 C2151 P51415 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 1460203 T4259 C2151 P51415 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 1460203 T4259 C2151 P51415 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:51395/_zgt/j/collection1/. core=collection1
[junit4:junit4] 2> 1460203 T4259 C2151 P51415 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1460205 T4163 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C2152 name=collection1 org.apache.solr.core.SolrCore@52ae2a57 url=http://127.0.0.1:51395/_zgt/j/collection1 node=127.0.0.1:51395__zgt%2Fj C2152_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:51395__zgt%2Fj, base_url=http://127.0.0.1:51395/_zgt/j, leader=true}
[junit4:junit4] 2> 1460210 T4212 C2152 P51395 oasc.SolrCore.execute [collection1] webapp=/_zgt/j path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 1460214 T4213 C2152 P51395 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1460217 T4213 C2152 P51395 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4006bc76 lockFactory=org.apache.lucene.store.NativeFSLockFactory@750ff575),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1460218 T4213 C2152 P51395 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1460218 T4213 C2152 P51395 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4006bc76 lockFactory=org.apache.lucene.store.NativeFSLockFactory@750ff575),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4006bc76 lockFactory=org.apache.lucene.store.NativeFSLockFactory@750ff575),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1460219 T4213 C2152 P51395 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1460220 T4213 C2152 P51395 oass.SolrIndexSearcher.<init> Opening Searcher@36ba9e86 realtime
[junit4:junit4] 2> 1460220 T4213 C2152 P51395 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1460220 T4213 C2152 P51395 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 1460221 T4259 C2151 P51415 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1460221 T4259 C2151 P51415 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 1460223 T4214 C2152 P51395 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1460224 T4214 C2152 P51395 oasc.SolrCore.execute [collection1] webapp=/_zgt/j path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 1460224 T4259 C2151 P51415 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 1460224 T4259 C2151 P51415 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 1460225 T4259 C2151 P51415 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 1460227 T4214 C2152 P51395 oasc.SolrCore.execute [collection1] webapp=/_zgt/j path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 1460227 T4259 C2151 P51415 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 1460228 T4259 C2151 P51415 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty4/index.20130618163516345
[junit4:junit4] 2> 1460229 T4259 C2151 P51415 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@709534db lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b9b66a0) fullCopy=false
[junit4:junit4] 2> 1460231 T4214 C2152 P51395 oasc.SolrCore.execute [collection1] webapp=/_zgt/j path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 1460232 T4259 C2151 P51415 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 1460233 T4259 C2151 P51415 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 1460233 T4259 C2151 P51415 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 1460235 T4259 C2151 P51415 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7564e83e lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b98a787),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7564e83e lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b98a787),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1460235 T4259 C2151 P51415 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1460235 T4259 C2151 P51415 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 1460236 T4259 C2151 P51415 oass.SolrIndexSearcher.<init> Opening Searcher@64c5fc85 main
[junit4:junit4] 2> 1460237 T4258 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@64c5fc85 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1460238 T4259 C2151 P51415 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty4/index.20130618163516345 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty4/index.20130618163516345;done=true>>]
[junit4:junit4] 2> 1460238 T4259 C2151 P51415 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty4/index.20130618163516345
[junit4:junit4] 2> 1460238 T4259 C2151 P51415 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty4/index.20130618163516345
[junit4:junit4] 2> 1460238 T4259 C2151 P51415 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 1460239 T4259 C2151 P51415 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 1460239 T4259 C2151 P51415 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1460239 T4259 C2151 P51415 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1460241 T4259 C2151 P51415 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 1460342 T4186 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1460343 T4186 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51415__zgt%2Fj",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51415/_zgt/j"}
[junit4:junit4] 2> 1460355 T4254 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1460356 T4221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1460356 T4205 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1460356 T4237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1460356 T4185 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1460356 T4192 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1461207 T4163 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1461208 T4163 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C2153 name=collection1 org.apache.solr.core.SolrCore@a7dc2c3 url=http://127.0.0.1:39731/_zgt/j/collection1 node=127.0.0.1:39731__zgt%2Fj C2153_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:39731__zgt%2Fj, base_url=http://127.0.0.1:39731/_zgt/j, leader=true}
[junit4:junit4] 2> 1461217 T4175 C2153 P39731 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1461221 T4175 C2153 P39731 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@30f848c6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3de53383),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1461221 T4175 C2153 P39731 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1461222 T4175 C2153 P39731 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@30f848c6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3de53383),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@30f848c6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3de53383),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1461223 T4175 C2153 P39731 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1461223 T4175 C2153 P39731 oass.SolrIndexSearcher.<init> Opening Searcher@75669877 main
[junit4:junit4] 2> 1461224 T4175 C2153 P39731 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1461225 T4189 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@75669877 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1461225 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 8
[junit4:junit4] 2> C2149_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:51404__zgt%2Fj, base_url=http://127.0.0.1:51404/_zgt/j}
[junit4:junit4] 2> 1461243 T4227 C2149 P51404 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:51389/_zgt/j/collection1/, StdNode: http://127.0.0.1:51404/_zgt/j/collection1/, StdNode: http://127.0.0.1:51395/_zgt/j/collection1/, StdNode: http://127.0.0.1:51415/_zgt/j/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 1461246 T4199 C2150 P51389 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1461246 T4215 C2152 P51395 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1461247 T4199 C2150 P51389 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@42817fe5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@60a6d3fc),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@42817fe5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@60a6d3fc),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1461247 T4228 C2149 P51404 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1461248 T4199 C2150 P51389 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1461247 T4215 C2152 P51395 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4006bc76 lockFactory=org.apache.lucene.store.NativeFSLockFactory@750ff575),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4006bc76 lockFactory=org.apache.lucene.store.NativeFSLockFactory@750ff575),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1461249 T4199 C2150 P51389 oass.SolrIndexSearcher.<init> Opening Searcher@73146ffe main
[junit4:junit4] 2> 1461248 T4228 C2149 P51404 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@70fae8b2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@45a324b1),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@70fae8b2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@45a324b1),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1461250 T4199 C2150 P51389 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1461249 T4215 C2152 P51395 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1461250 T4207 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@73146ffe main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1461250 T4228 C2149 P51404 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1461252 T4199 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 1461251 T4215 C2152 P51395 oass.SolrIndexSearcher.<init> Opening Searcher@5142347 main
[junit4:junit4] 2> 1461253 T4228 C2149 P51404 oass.SolrIndexSearcher.<init> Opening Searcher@368cb2cf main
[junit4:junit4] 2> C2151_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:51415__zgt%2Fj, base_url=http://127.0.0.1:51415/_zgt/j}
[junit4:junit4] 2> 1461254 T4244 C2151 P51415 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1461255 T4223 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5142347 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1461255 T4244 C2151 P51415 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7564e83e lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b98a787),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7564e83e lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b98a787),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1461254 T4215 C2152 P51395 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1461256 T4244 C2151 P51415 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1461255 T4239 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@368cb2cf main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1461255 T4228 C2149 P51404 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1461257 T4244 C2151 P51415 oass.SolrIndexSearcher.<init> Opening Searcher@76d6f31f main
[junit4:junit4] 2> 1461256 T4215 C2152 P51395 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4] 2> 1461258 T4244 C2151 P51415 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1461259 T4258 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@76d6f31f main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1461257 T4228 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4] 2> 1461259 T4244 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> 1461260 T4227 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 17
[junit4:junit4] 2> 1461261 T4163 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1461264 T4200 C2150 P51389 oasc.SolrCore.execute [collection1] webapp=/_zgt/j path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 1461265 T4229 C2149 P51404 oasc.SolrCore.execute [collection1] webapp=/_zgt/j path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 1461267 T4216 C2152 P51395 oasc.SolrCore.execute [collection1] webapp=/_zgt/j path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 1461269 T4245 C2151 P51415 oasc.SolrCore.execute [collection1] webapp=/_zgt/j path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 1463274 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1438217739319640064)} 0 2
[junit4:junit4] 2> 1463284 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={update.distrib=FROMLEADER&_version_=-1438217739323834368&update.from=http://127.0.0.1:51389/_zgt/j/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438217739323834368)} 0 2
[junit4:junit4] 2> 1463287 T4246 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={update.distrib=FROMLEADER&_version_=-1438217739330125824&update.from=http://127.0.0.1:51395/_zgt/j/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438217739330125824)} 0 2
[junit4:junit4] 2> 1463288 T4217 C2152 P51395 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1438217739330125824)} 0 6
[junit4:junit4] 2> 1463288 T4201 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1438217739323834368)} 0 12
[junit4:junit4] 2> 1463292 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[0 (1438217739339563008)]} 0 1
[junit4:junit4] 2> 1463297 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438217739343757312)]} 0 0
[junit4:junit4] 2> 1463298 T4195 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51395/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[0 (1438217739343757312)]} 0 3
[junit4:junit4] 2> 1463299 T4211 C2152 P51395 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[0]} 0 6
[junit4:junit4] 2> 1463302 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[1 (1438217739351097344)]} 0 0
[junit4:junit4] 2> 1463310 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438217739356340224)]} 0 0
[junit4:junit4] 2> 1463311 T4199 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51404/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1438217739356340224)]} 0 4
[junit4:junit4] 2> 1463312 T4227 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[1]} 0 7
[junit4:junit4] 2> 1463314 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[2 (1438217739363680256)]} 0 0
[junit4:junit4] 2> 1463320 T4246 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51395/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1438217739366825984)]} 0 0
[junit4:junit4] 2> 1463321 T4211 C2152 P51395 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[2 (1438217739366825984)]} 0 4
[junit4:junit4] 2> 1463324 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[3 (1438217739374166016)]} 0 0
[junit4:junit4] 2> 1463331 T4246 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51395/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1438217739379408896)]} 0 0
[junit4:junit4] 2> 1463332 T4217 C2152 P51395 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[3 (1438217739379408896)]} 0 3
[junit4:junit4] 2> 1463333 T4201 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[3]} 0 6
[junit4:junit4] 2> 1463335 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[4 (1438217739385700352)]} 0 0
[junit4:junit4] 2> 1463342 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1438217739390943232)]} 0 0
[junit4:junit4] 2> 1463343 T4199 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51404/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[4 (1438217739390943232)]} 0 3
[junit4:junit4] 2> 1463344 T4227 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[4]} 0 6
[junit4:junit4] 2> 1463346 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[5 (1438217739397234688)]} 0 0
[junit4:junit4] 2> 1463352 T4246 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51395/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[5 (1438217739401428992)]} 0 0
[junit4:junit4] 2> 1463353 T4213 C2152 P51395 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51415/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[5 (1438217739401428992)]} 0 3
[junit4:junit4] 2> 1463354 T4247 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[5]} 0 6
[junit4:junit4] 2> 1463357 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[6 (1438217739408769024)]} 0 0
[junit4:junit4] 2> 1463364 T4246 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51395/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[6 (1438217739414011904)]} 0 0
[junit4:junit4] 2> 1463365 T4217 C2152 P51395 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[6 (1438217739414011904)]} 0 3
[junit4:junit4] 2> 1463366 T4201 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[6]} 0 6
[junit4:junit4] 2> 1463368 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[7 (1438217739420303360)]} 0 0
[junit4:junit4] 2> 1463373 T4246 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51395/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[7 (1438217739423449088)]} 0 0
[junit4:junit4] 2> 1463374 T4211 C2152 P51395 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[7 (1438217739423449088)]} 0 3
[junit4:junit4] 2> 1463377 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[8 (1438217739429740544)]} 0 0
[junit4:junit4] 2> 1463383 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1438217739432886272)]} 0 0
[junit4:junit4] 2> 1463384 T4201 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[8 (1438217739432886272)]} 0 4
[junit4:junit4] 2> 1463387 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[9 (1438217739440226304)]} 0 0
[junit4:junit4] 2> 1463392 T4246 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51395/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[9 (1438217739443372032)]} 0 0
[junit4:junit4] 2> 1463393 T4211 C2152 P51395 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[9 (1438217739443372032)]} 0 3
[junit4:junit4] 2> 1463396 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[10 (1438217739449663488)]} 0 0
[junit4:junit4] 2> 1463403 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1438217739454906368)]} 0 0
[junit4:junit4] 2> 1463404 T4199 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51404/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[10 (1438217739454906368)]} 0 3
[junit4:junit4] 2> 1463405 T4227 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[10]} 0 6
[junit4:junit4] 2> 1463407 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[11 (1438217739461197824)]} 0 0
[junit4:junit4] 2> 1463412 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[11 (1438217739464343552)]} 0 0
[junit4:junit4] 2> 1463413 T4201 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[11 (1438217739464343552)]} 0 3
[junit4:junit4] 2> 1463416 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[12 (1438217739470635008)]} 0 0
[junit4:junit4] 2> 1463423 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[12 (1438217739474829312)]} 0 0
[junit4:junit4] 2> 1463424 T4197 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51415/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[12 (1438217739474829312)]} 0 4
[junit4:junit4] 2> 1463425 T4247 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[12]} 0 6
[junit4:junit4] 2> 1463427 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[13 (1438217739482169344)]} 0 0
[junit4:junit4] 2> 1463435 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[13 (1438217739487412224)]} 0 0
[junit4:junit4] 2> 1463436 T4199 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51404/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[13 (1438217739487412224)]} 0 4
[junit4:junit4] 2> 1463437 T4227 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[13]} 0 7
[junit4:junit4] 2> 1463439 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[14 (1438217739494752256)]} 0 0
[junit4:junit4] 2> 1463446 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[14 (1438217739499995136)]} 0 0
[junit4:junit4] 2> 1463447 T4197 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51415/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[14 (1438217739499995136)]} 0 3
[junit4:junit4] 2> 1463448 T4247 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[14]} 0 6
[junit4:junit4] 2> 1463450 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[15 (1438217739506286592)]} 0 0
[junit4:junit4] 2> 1463458 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[15 (1438217739511529472)]} 0 0
[junit4:junit4] 2> 1463459 T4199 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51404/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[15 (1438217739511529472)]} 0 4
[junit4:junit4] 2> 1463460 T4227 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[15]} 0 7
[junit4:junit4] 2> 1463462 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[16 (1438217739518869504)]} 0 0
[junit4:junit4] 2> 1463468 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[16 (1438217739522015232)]} 0 0
[junit4:junit4] 2> 1463469 T4201 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[16 (1438217739522015232)]} 0 4
[junit4:junit4] 2> 1463472 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[17 (1438217739529355264)]} 0 0
[junit4:junit4] 2> 1463479 T4246 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51395/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[17 (1438217739534598144)]} 0 0
[junit4:junit4] 2> 1463480 T4213 C2152 P51395 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51415/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[17 (1438217739534598144)]} 0 3
[junit4:junit4] 2> 1463481 T4247 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[17]} 0 6
[junit4:junit4] 2> 1463484 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[18 (1438217739541938176)]} 0 0
[junit4:junit4] 2> 1463491 T4246 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51395/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[18 (1438217739547181056)]} 0 0
[junit4:junit4] 2> 1463492 T4213 C2152 P51395 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51415/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[18 (1438217739547181056)]} 0 3
[junit4:junit4] 2> 1463493 T4247 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[18]} 0 6
[junit4:junit4] 2> 1463495 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[19 (1438217739553472512)]} 0 0
[junit4:junit4] 2> 1463502 T4246 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51395/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[19 (1438217739558715392)]} 0 0
[junit4:junit4] 2> 1463503 T4215 C2152 P51395 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51404/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[19 (1438217739558715392)]} 0 3
[junit4:junit4] 2> 1463504 T4227 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[19]} 0 6
[junit4:junit4] 2> 1463506 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[20 (1438217739565006848)]} 0 0
[junit4:junit4] 2> 1463516 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[20 (1438217739571298304)]} 0 1
[junit4:junit4] 2> 1463517 T4199 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51404/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[20 (1438217739571298304)]} 0 5
[junit4:junit4] 2> 1463517 T4227 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[20]} 0 7
[junit4:junit4] 2> 1463520 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[21 (1438217739579686912)]} 0 0
[junit4:junit4] 2> 1463527 T4246 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51395/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[21 (1438217739584929792)]} 0 0
[junit4:junit4] 2> 1463528 T4215 C2152 P51395 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51404/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[21 (1438217739584929792)]} 0 3
[junit4:junit4] 2> 1463529 T4227 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[21]} 0 6
[junit4:junit4] 2> 1463531 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[22 (1438217739591221248)]} 0 0
[junit4:junit4] 2> 1463539 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[22 (1438217739596464128)]} 0 1
[junit4:junit4] 2> 1463539 T4199 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51404/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[22 (1438217739596464128)]} 0 3
[junit4:junit4] 2> 1463540 T4227 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[22]} 0 6
[junit4:junit4] 2> 1463543 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[23 (1438217739603804160)]} 0 0
[junit4:junit4] 2> 1463551 T4246 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51395/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[23 (1438217739609047040)]} 0 0
[junit4:junit4] 2> 1463552 T4217 C2152 P51395 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[23 (1438217739609047040)]} 0 4
[junit4:junit4] 2> 1463553 T4201 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[23]} 0 7
[junit4:junit4] 2> 1463556 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[24 (1438217739617435648)]} 0 0
[junit4:junit4] 2> 1463564 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[24 (1438217739622678528)]} 0 0
[junit4:junit4] 2> 1463565 T4197 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51415/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[24 (1438217739622678528)]} 0 4
[junit4:junit4] 2> 1463566 T4247 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[24]} 0 7
[junit4:junit4] 2> 1463569 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[25 (1438217739631067136)]} 0 0
[junit4:junit4] 2> 1463577 T4246 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51395/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[25 (1438217739636310016)]} 0 0
[junit4:junit4] 2> 1463578 T4213 C2152 P51395 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51415/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[25 (1438217739636310016)]} 0 4
[junit4:junit4] 2> 1463579 T4247 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[25]} 0 7
[junit4:junit4] 2> 1463582 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[26 (1438217739644698624)]} 0 0
[junit4:junit4] 2> 1463589 T4246 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51395/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[26 (1438217739649941504)]} 0 0
[junit4:junit4] 2> 1463590 T4213 C2152 P51395 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51415/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[26 (1438217739649941504)]} 0 3
[junit4:junit4] 2> 1463591 T4247 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[26]} 0 6
[junit4:junit4] 2> 1463593 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[27 (1438217739656232960)]} 0 0
[junit4:junit4] 2> 1463601 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[27 (1438217739661475840)]} 0 1
[junit4:junit4] 2> 1463602 T4195 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51395/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[27 (1438217739661475840)]} 0 4
[junit4:junit4] 2> 1463603 T4211 C2152 P51395 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[27]} 0 7
[junit4:junit4] 2> 1463607 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[28 (1438217739670913024)]} 0 1
[junit4:junit4] 2> 1463618 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[28 (1438217739678253056)]} 0 0
[junit4:junit4] 2> 1463619 T4197 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51415/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[28 (1438217739678253056)]} 0 5
[junit4:junit4] 2> 1463619 T4247 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[28]} 0 8
[junit4:junit4] 2> 1463622 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[29 (1438217739686641664)]} 0 0
[junit4:junit4] 2> 1463628 T4246 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51395/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[29 (1438217739689787392)]} 0 0
[junit4:junit4] 2> 1463629 T4211 C2152 P51395 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[29 (1438217739689787392)]} 0 4
[junit4:junit4] 2> 1463632 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[30 (1438217739697127424)]} 0 0
[junit4:junit4] 2> 1463639 T4246 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51395/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[30 (1438217739702370304)]} 0 0
[junit4:junit4] 2> 1463640 T4217 C2152 P51395 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[30 (1438217739702370304)]} 0 3
[junit4:junit4] 2> 1463641 T4201 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[30]} 0 6
[junit4:junit4] 2> 1463643 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[31 (1438217739708661760)]} 0 0
[junit4:junit4] 2> 1463648 T4246 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51395/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[31 (1438217739711807488)]} 0 0
[junit4:junit4] 2> 1463649 T4211 C2152 P51395 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[31 (1438217739711807488)]} 0 3
[junit4:junit4] 2> 1463652 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[32 (1438217739718098944)]} 0 0
[junit4:junit4] 2> 1463660 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[32 (1438217739723341824)]} 0 0
[junit4:junit4] 2> 1463661 T4197 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51415/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[32 (1438217739723341824)]} 0 4
[junit4:junit4] 2> 1463662 T4247 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[32]} 0 7
[junit4:junit4] 2> 1463664 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[33 (1438217739730681856)]} 0 0
[junit4:junit4] 2> 1463671 T4246 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51395/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[33 (1438217739733827584)]} 0 1
[junit4:junit4] 2> 1463671 T4211 C2152 P51395 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[33 (1438217739733827584)]} 0 4
[junit4:junit4] 2> 1463674 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[34 (1438217739741167616)]} 0 0
[junit4:junit4] 2> 1463679 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[34 (1438217739744313344)]} 0 0
[junit4:junit4] 2> 1463680 T4201 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[34 (1438217739744313344)]} 0 3
[junit4:junit4] 2> 1463683 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[35 (1438217739750604800)]} 0 0
[junit4:junit4] 2> 1463690 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[35 (1438217739755847680)]} 0 0
[junit4:junit4] 2> 1463691 T4199 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51404/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[35 (1438217739755847680)]} 0 3
[junit4:junit4] 2> 1463692 T4227 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[35]} 0 6
[junit4:junit4] 2> 1463695 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[36 (1438217739762139136)]} 0 1
[junit4:junit4] 2> 1463701 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[36 (1438217739767382016)]} 0 0
[junit4:junit4] 2> 1463702 T4195 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51395/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[36 (1438217739767382016)]} 0 3
[junit4:junit4] 2> 1463703 T4211 C2152 P51395 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[36]} 0 6
[junit4:junit4] 2> 1463705 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[37 (1438217739773673472)]} 0 0
[junit4:junit4] 2> 1463713 T4246 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51395/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[37 (1438217739778916352)]} 0 0
[junit4:junit4] 2> 1463714 T4215 C2152 P51395 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51404/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[37 (1438217739778916352)]} 0 4
[junit4:junit4] 2> 1463715 T4227 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[37]} 0 7
[junit4:junit4] 2> 1463717 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[38 (1438217739786256384)]} 0 0
[junit4:junit4] 2> 1463724 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[38 (1438217739791499264)]} 0 0
[junit4:junit4] 2> 1463725 T4197 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51415/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[38 (1438217739791499264)]} 0 3
[junit4:junit4] 2> 1463726 T4247 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[38]} 0 6
[junit4:junit4] 2> 1463728 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[39 (1438217739797790720)]} 0 0
[junit4:junit4] 2> 1463735 T4246 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51395/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[39 (1438217739803033600)]} 0 0
[junit4:junit4] 2> 1463736 T4217 C2152 P51395 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[39 (1438217739803033600)]} 0 3
[junit4:junit4] 2> 1463737 T4201 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[39]} 0 6
[junit4:junit4] 2> 1463739 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[40 (1438217739809325056)]} 0 0
[junit4:junit4] 2> 1463744 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[40 (1438217739812470784)]} 0 0
[junit4:junit4] 2> 1463745 T4201 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[40 (1438217739812470784)]} 0 3
[junit4:junit4] 2> 1463748 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[41 (1438217739818762240)]} 0 0
[junit4:junit4] 2> 1463755 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[41 (1438217739824005120)]} 0 0
[junit4:junit4] 2> 1463756 T4199 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51404/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[41 (1438217739824005120)]} 0 3
[junit4:junit4] 2> 1463757 T4227 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[41]} 0 6
[junit4:junit4] 2> 1463759 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[42 (1438217739830296576)]} 0 0
[junit4:junit4] 2> 1463766 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[42 (1438217739835539456)]} 0 0
[junit4:junit4] 2> 1463767 T4197 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51415/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[42 (1438217739835539456)]} 0 3
[junit4:junit4] 2> 1463768 T4247 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[42]} 0 6
[junit4:junit4] 2> 1463770 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[43 (1438217739841830912)]} 0 0
[junit4:junit4] 2> 1463777 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[43 (1438217739847073792)]} 0 0
[junit4:junit4] 2> 1463778 T4199 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51404/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[43 (1438217739847073792)]} 0 3
[junit4:junit4] 2> 1463779 T4227 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[43]} 0 6
[junit4:junit4] 2> 1463781 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[44 (1438217739853365248)]} 0 0
[junit4:junit4] 2> 1463786 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[44 (1438217739856510976)]} 0 0
[junit4:junit4] 2> 1463787 T4201 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[44 (1438217739856510976)]} 0 3
[junit4:junit4] 2> 1463790 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[45 (1438217739862802432)]} 0 0
[junit4:junit4] 2> 1463797 T4246 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51395/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[45 (1438217739868045312)]} 0 0
[junit4:junit4] 2> 1463798 T4213 C2152 P51395 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51415/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[45 (1438217739868045312)]} 0 3
[junit4:junit4] 2> 1463799 T4247 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[45]} 0 6
[junit4:junit4] 2> 1463801 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[46 (1438217739874336768)]} 0 0
[junit4:junit4] 2> 1463808 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[46 (1438217739879579648)]} 0 0
[junit4:junit4] 2> 1463809 T4199 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51404/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[46 (1438217739879579648)]} 0 3
[junit4:junit4] 2> 1463810 T4227 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[46]} 0 6
[junit4:junit4] 2> 1463812 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[47 (1438217739885871104)]} 0 0
[junit4:junit4] 2> 1463817 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[47 (1438217739889016832)]} 0 0
[junit4:junit4] 2> 1463818 T4201 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[47 (1438217739889016832)]} 0 3
[junit4:junit4] 2> 1463821 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[48 (1438217739895308288)]} 0 0
[junit4:junit4] 2> 1463828 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[48 (1438217739900551168)]} 0 0
[junit4:junit4] 2> 1463829 T4195 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51395/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[48 (1438217739900551168)]} 0 3
[junit4:junit4] 2> 1463830 T4211 C2152 P51395 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[48]} 0 6
[junit4:junit4] 2> 1463832 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[49 (1438217739906842624)]} 0 0
[junit4:junit4] 2> 1463839 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[49 (1438217739912085504)]} 0 0
[junit4:junit4] 2> 1463840 T4199 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51404/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[49 (1438217739912085504)]} 0 3
[junit4:junit4] 2> 1463841 T4227 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[49]} 0 6
[junit4:junit4] 2> 1463843 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[50 (1438217739918376960)]} 0 0
[junit4:junit4] 2> 1463850 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[50 (1438217739923619840)]} 0 0
[junit4:junit4] 2> 1463851 T4199 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51404/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[50 (1438217739923619840)]} 0 3
[junit4:junit4] 2> 1463852 T4227 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[50]} 0 6
[junit4:junit4] 2> 1463856 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[51 (1438217739930959872)]} 0 1
[junit4:junit4] 2> 1463862 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[51 (1438217739935154176)]} 0 0
[junit4:junit4] 2> 1463863 T4201 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[51 (1438217739935154176)]} 0 4
[junit4:junit4] 2> 1463866 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[52 (1438217739942494208)]} 0 0
[junit4:junit4] 2> 1463874 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[52 (1438217739947737088)]} 0 0
[junit4:junit4] 2> 1463875 T4197 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51415/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[52 (1438217739947737088)]} 0 4
[junit4:junit4] 2> 1463876 T4247 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[52]} 0 7
[junit4:junit4] 2> 1463879 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[53 (1438217739956125696)]} 0 0
[junit4:junit4] 2> 1463886 T4246 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51395/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[53 (1438217739961368576)]} 0 0
[junit4:junit4] 2> 1463887 T4215 C2152 P51395 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51404/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[53 (1438217739961368576)]} 0 3
[junit4:junit4] 2> 1463888 T4227 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[53]} 0 6
[junit4:junit4] 2> 1463890 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[54 (1438217739967660032)]} 0 0
[junit4:junit4] 2> 1463897 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[54 (1438217739972902912)]} 0 0
[junit4:junit4] 2> 1463898 T4197 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51415/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[54 (1438217739972902912)]} 0 3
[junit4:junit4] 2> 1463898 T4247 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[54]} 0 5
[junit4:junit4] 2> 1463901 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[55 (1438217739979194368)]} 0 0
[junit4:junit4] 2> 1463909 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[55 (1438217739984437248)]} 0 0
[junit4:junit4] 2> 1463910 T4199 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51404/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[55 (1438217739984437248)]} 0 4
[junit4:junit4] 2> 1463911 T4227 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[55]} 0 7
[junit4:junit4] 2> 1463913 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[56 (1438217739991777280)]} 0 0
[junit4:junit4] 2> 1463918 T4246 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51395/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[56 (1438217739994923008)]} 0 0
[junit4:junit4] 2> 1463919 T4211 C2152 P51395 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[56 (1438217739994923008)]} 0 3
[junit4:junit4] 2> 1463922 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[57 (1438217740001214464)]} 0 0
[junit4:junit4] 2> 1463927 T4246 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51395/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[57 (1438217740004360192)]} 0 0
[junit4:junit4] 2> 1463928 T4211 C2152 P51395 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[57 (1438217740004360192)]} 0 3
[junit4:junit4] 2> 1463932 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[58 (1438217740010651648)]} 0 1
[junit4:junit4] 2> 1463939 T4246 C2151 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51395/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[58 (1438217740016943104)]} 0 0
[junit4:junit4] 2> 1463940 T4217 C2152 P51395 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[58 (1438217740016943104)]} 0 3
[junit4:junit4] 2> 1463941 T4201 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[58]} 0 6
[junit4:junit4] 2> 1463943 T4175 C2153 P39731 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[59 (1438217740023234560)]} 0 0
[junit4:junit4] 2> 1463950 T4230 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51389/_zgt/j/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[59 (1438217740028477440)]} 0 0
[junit4:junit4] 2> 1463951 T4199 C2150 P51389 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={distrib.from=http://127.0.0.1:51404/_zgt/j/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[59 (1438217740028477440)]} 0 3
[junit4:junit4] 2> 1463952 T4227 C2149 P51404 oasup.LogUpdateProcessor.finish [collection1] webapp=/_zgt/j path=/update params={wt=javabin&version=2} {add=[59]} 0 6
[...truncated too long message...]
on request.
[junit4:junit4] 2> 1688600 T4254 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 1688600 T4254 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty4/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty4/index;done=false>>]
[junit4:junit4] 2> 1688600 T4254 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty4/index
[junit4:junit4] 2> 1688601 T4254 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty4 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty4;done=false>>]
[junit4:junit4] 2> 1688601 T4254 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371591288176/jetty4
[junit4:junit4] 2> 1688601 T4254 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 1688602 T4254 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard2/election/89888606662426636-5-n_0000000001
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4] 2> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:152)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:149)
[junit4:junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:149)
[junit4:junit4] 2> at org.apache.solr.cloud.ElectionContext.cancelElection(ElectionContext.java:63)
[junit4:junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:260)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:156)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:100)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4] 2>
[junit4:junit4] 2> 1688602 T4254 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1688603 T4254 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 1688603 T4254 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 1688603 T4254 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1688603 T4254 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 1688604 T4254 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:235)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:232)
[junit4:junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:232)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4] 2>
[junit4:junit4] 2> NOTE: test params are: codec=Lucene41, sim=DefaultSimilarity, locale=iw_IL, timezone=US/Indiana-Starke
[junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=2,free=289874768,total=454033408
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestLFUCache, TermsComponentTest, TermVectorComponentTest, TestSchemaSimilarityResource, TestLRUCache, TestIBSimilarityFactory, SliceStateTest, CacheHeaderTest, CachingDirectoryFactoryTest, TestCloudManagedSchemaAddField, TestSolrIndexConfig, TestExtendedDismaxParser, ResourceLoaderTest, SpatialFilterTest, LegacyHTMLStripCharFilterTest, TestDocumentBuilder, TestQueryUtils, DistributedTermsComponentTest, TestJoin, TestXIncludeConfig, TestPseudoReturnFields, TestCharFilters, ZkControllerTest, DirectUpdateHandlerOptimizeTest, TestStressRecovery, DateFieldTest, QueryParsingTest, UnloadDistributedZkTest, SuggesterTest, TestTrie, BasicDistributedZkTest, SolrCoreTest, TestReversedWildcardFilterFactory, TestClassNameShortening, TestPropInjectDefaults, SyncSliceTest, TestPluginEnable, TestCoreContainer, LeaderElectionTest, BadComponentTest, TestFaceting, DirectSolrConnectionTest, DocValuesTest, ShowFileRequestHandlerTest, SuggesterFSTTest, AutoCommitTest, TestGroupingSearch, TestPerFieldSimilarity, ReturnFieldsTest, TestPhraseSuggestions, TestUniqueKeyFieldResource, SOLR749Test, CircularListTest, TestSweetSpotSimilarityFactory, TestFastWriter, TestStressVersions, ZkCLITest, TestReplicationHandler, SimpleFacetsTest, TestSurroundQueryParser, TestQuerySenderListener, WordBreakSolrSpellCheckerTest, PreAnalyzedFieldTest, PrimitiveFieldTypeTest, CoreAdminHandlerTest, QueryElevationComponentTest, BasicFunctionalityTest, FieldMutatingUpdateProcessorTest, TestSort, TestFuzzyAnalyzedSuggestions, UpdateRequestProcessorFactoryTest, QueryResultKeyTest, BasicDistributedZk2Test, OpenCloseCoreStressTest, RecoveryZkTest, FullSolrCloudDistribCmdsTest, ShardRoutingCustomTest, SliceStateUpdateTest, TestDistributedSearch, TestRealTimeGet, TestRangeQuery, PeerSyncTest, BadIndexSchemaTest, TestFunctionQuery, TestBadConfig, OverseerCollectionProcessorTest, CurrencyFieldOpenExchangeTest, DistributedQueryElevationComponentTest, SolrIndexSplitterTest, SignatureUpdateProcessorFactoryTest, SuggesterTSTTest, TestCSVLoader, NoCacheHeaderTest, TestWordDelimiterFilterFactory, DefaultValueUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, DistanceFunctionTest, SolrInfoMBeanTest, IndexBasedSpellCheckerTest, DisMaxRequestHandlerTest, StandardRequestHandlerTest, TestWriterPerf, TestOmitPositions, RequestHandlersTest, TestValueSourceCache, TestIndexingPerformance, FieldAnalysisRequestHandlerTest, FastVectorHighlighterTest, RegexBoostProcessorTest, IndexSchemaTest, MBeansHandlerTest, JsonLoaderTest, TestPHPSerializedResponseWriter, CSVRequestHandlerTest, CoreContainerCoreInitFailuresTest, TestPropInject, UpdateParamsTest, AlternateDirectoryTest, CopyFieldTest, TestMergePolicyConfig, TestSolrDeletionPolicy2, TestBinaryField, MinimalSchemaTest, OutputWriterTest, TestSolrCoreProperties, NotRequiredUniqueKeyTest, TestCodecSupport, EchoParamsTest, TestLMDirichletSimilarityFactory, TestDefaultSimilarityFactory, TestFastOutputStream, OpenExchangeRatesOrgProviderTest, URLClassifyProcessorTest, DateMathParserTest, TestSolrXMLSerializer, TestSuggestSpellingConverter, RAMDirectoryFactoryTest, SystemInfoHandlerTest, FileUtilsTest, TestRTGBase, ChaosMonkeyShardSplitTest]
[junit4:junit4] Completed on J0 in 257.11s, 1 test, 1 error <<< FAILURES!
[...truncated 51 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:386: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:366: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:181: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:437: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1243: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:887: There were test failures: 297 suites, 1250 tests, 1 error, 13 ignored (7 assumptions)
Total time: 51 minutes 56 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4074 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4074/
4 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
Error Message:
ERROR: SolrIndexSearcher opens=40 closes=39
Stack Trace:
java.lang.AssertionError: ERROR: SolrIndexSearcher opens=40 closes=39
at __randomizedtesting.SeedInfo.seed([BDAD784777C68360]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:254)
at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:102)
at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:700)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
FAILED: org.apache.solr.cloud.ChaosMonkeyShardSplitTest.testDistribSearch
Error Message:
Server at http://127.0.0.1:13797 returned non ok status:500, message:Server Error
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:13797 returned non ok status:500, message:Server Error
at __randomizedtesting.SeedInfo.seed([BDAD784777C68360:3C4BF65F0099E35C]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
at org.apache.solr.cloud.ChaosMonkeyShardSplitTest.doTest(ChaosMonkeyShardSplitTest.java:136)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
FAILED: org.apache.solr.cloud.RecoveryZkTest.testDistribSearch
Error Message:
expected:<2110> but was:<62>
Stack Trace:
java.lang.AssertionError: expected:<2110> but was:<62>
at __randomizedtesting.SeedInfo.seed([BDAD784777C68360:3C4BF65F0099E35C]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.RecoveryZkTest.doTest(RecoveryZkTest.java:108)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
FAILED: org.apache.solr.cloud.ShardSplitTest.testDistribSearch
Error Message:
Server at http://127.0.0.1:22097/_nx returned non ok status:500, message:Server Error
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:22097/_nx returned non ok status:500, message:Server Error
at __randomizedtesting.SeedInfo.seed([BDAD784777C68360:3C4BF65F0099E35C]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
at org.apache.solr.cloud.ShardSplitTest.doTest(ShardSplitTest.java:150)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9292 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.RecoveryZkTest
[junit4:junit4] 2> 84662 T124 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /yu/
[junit4:junit4] 2> 84669 T124 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./solrtest-RecoveryZkTest-1371557346779
[junit4:junit4] 2> 84681 T124 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 84691 T125 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 84892 T124 oasc.ZkTestServer.run start zk server on port:63934
[junit4:junit4] 2> 85192 T124 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 85242 T126 oazs.NIOServerCnxn.doIO WARN Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running
[junit4:junit4] 2> 86656 T129 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6f40f6c5 name:ZooKeeperConnection Watcher:127.0.0.1:63934 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 86657 T124 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 86663 T124 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 86771 T124 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 86774 T133 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4b92c9 name:ZooKeeperConnection Watcher:127.0.0.1:63934/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 86774 T124 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 86787 T124 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 86794 T124 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 86799 T124 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 86804 T124 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 86809 T124 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 86811 T124 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 86825 T124 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 86826 T124 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 86933 T124 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 86934 T124 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 86940 T124 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 86941 T124 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 86947 T124 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 86947 T124 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 86953 T124 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 86954 T124 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 86960 T124 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 86961 T124 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 86967 T124 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 86967 T124 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 86973 T124 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 86974 T124 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 87261 T124 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 87265 T124 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:63938
[junit4:junit4] 2> 87266 T124 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 87267 T124 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 87267 T124 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1371557349090
[junit4:junit4] 2> 87268 T124 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1371557349090/solr.xml
[junit4:junit4] 2> 87268 T124 oasc.CoreContainer.<init> New CoreContainer 1537506882
[junit4:junit4] 2> 87269 T124 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1371557349090/'
[junit4:junit4] 2> 87270 T124 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1371557349090/'
[junit4:junit4] 2> 87405 T124 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 87406 T124 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 87407 T124 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 87408 T124 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 87408 T124 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 87409 T124 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 87409 T124 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 87410 T124 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 87411 T124 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 87411 T124 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 87432 T124 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 87433 T124 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:63934/solr
[junit4:junit4] 2> 87469 T124 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 87470 T124 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 87474 T144 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1671b3a1 name:ZooKeeperConnection Watcher:127.0.0.1:63934 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 87474 T124 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 87483 T124 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 87505 T124 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 87508 T146 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7655e80c name:ZooKeeperConnection Watcher:127.0.0.1:63934/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 87508 T124 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 87524 T124 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 87543 T124 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 87566 T124 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 87570 T124 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63938_yu
[junit4:junit4] 2> 87579 T124 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:63938_yu
[junit4:junit4] 2> 87586 T124 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 87620 T124 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 87625 T124 oasc.Overseer.start Overseer (id=89886382306230275-127.0.0.1:63938_yu-n_0000000000) starting
[junit4:junit4] 2> 87661 T124 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 87686 T148 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 87686 T124 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 87691 T124 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 87694 T124 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 87725 T147 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 87744 T149 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 87744 T149 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 89238 T147 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 89240 T147 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:63938_yu",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:63938/yu"}
[junit4:junit4] 2> 89240 T147 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 89279 T147 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 89309 T146 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 89750 T149 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1371557349090/collection1
[junit4:junit4] 2> 89750 T149 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 89752 T149 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 89752 T149 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 89755 T149 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1371557349090/collection1/'
[junit4:junit4] 2> 89757 T149 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-controljetty-1371557349090/collection1/lib/README' to classloader
[junit4:junit4] 2> 89757 T149 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-controljetty-1371557349090/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 89849 T149 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 89915 T149 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 90017 T149 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 90024 T149 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 90685 T149 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 90699 T149 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 90703 T149 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 90732 T149 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 90738 T149 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 90743 T149 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 90745 T149 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 90746 T149 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 90746 T149 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 90748 T149 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 90748 T149 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 90749 T149 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 90749 T149 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1371557349090/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1371557346779/control/data/
[junit4:junit4] 2> 90750 T149 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@790bc5c9
[junit4:junit4] 2> 90751 T149 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 90751 T149 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1371557346779/control/data
[junit4:junit4] 2> 90752 T149 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1371557346779/control/data/index/
[junit4:junit4] 2> 90752 T149 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1371557346779/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 90753 T149 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1371557346779/control/data/index
[junit4:junit4] 2> 90758 T149 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@36f7bfd8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@59d76e68),segFN=segments_1,generation=1}
[junit4:junit4] 2> 90758 T149 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 90831 T149 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 90831 T149 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 90832 T149 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 90833 T149 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 90833 T149 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 90834 T149 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 90834 T149 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 90835 T149 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 90835 T149 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 90850 T149 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 90857 T149 oass.SolrIndexSearcher.<init> Opening Searcher@f103666 main
[junit4:junit4] 2> 90858 T149 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.RecoveryZkTest-1371557346779/control/data/tlog
[junit4:junit4] 2> 90859 T149 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 90860 T149 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 90864 T150 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@f103666 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 90866 T149 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 90867 T149 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:63938/yu collection:control_collection shard:shard1
[junit4:junit4] 2> 90883 T149 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4] 2> 90903 T149 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 90925 T149 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 90935 T149 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 90936 T149 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 90942 T149 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:63938/yu/collection1/
[junit4:junit4] 2> 90942 T149 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 90942 T149 oasc.SyncStrategy.syncToMe http://127.0.0.1:63938/yu/collection1/ has no replicas
[junit4:junit4] 2> 90943 T149 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:63938/yu/collection1/
[junit4:junit4] 2> 90943 T149 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 92320 T147 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 92343 T146 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 92383 T149 oasc.ZkController.register We are http://127.0.0.1:63938/yu/collection1/ and leader is http://127.0.0.1:63938/yu/collection1/
[junit4:junit4] 2> 92383 T149 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:63938/yu
[junit4:junit4] 2> 92384 T149 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 92384 T149 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 92384 T149 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 92387 T149 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 92389 T124 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 92390 T124 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 92391 T124 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 92407 T124 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 92414 T124 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 92418 T153 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@58708a09 name:ZooKeeperConnection Watcher:127.0.0.1:63934/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 92418 T124 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 92420 T124 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 92427 T124 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 92688 T124 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 92692 T124 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:63942
[junit4:junit4] 2> 92693 T124 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 92693 T124 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 92694 T124 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1371557354537
[junit4:junit4] 2> 92694 T124 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1371557354537/solr.xml
[junit4:junit4] 2> 92695 T124 oasc.CoreContainer.<init> New CoreContainer 964863654
[junit4:junit4] 2> 92696 T124 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1371557354537/'
[junit4:junit4] 2> 92696 T124 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1371557354537/'
[junit4:junit4] 2> 92835 T124 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 92836 T124 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 92837 T124 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 92838 T124 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 92838 T124 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 92839 T124 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 92839 T124 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 92840 T124 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 92840 T124 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 92841 T124 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 92859 T124 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 92860 T124 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:63934/solr
[junit4:junit4] 2> 92861 T124 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 92862 T124 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 92866 T164 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13a6f0be name:ZooKeeperConnection Watcher:127.0.0.1:63934 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 92866 T124 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 92882 T124 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 92895 T124 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 92898 T166 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3228ffdf name:ZooKeeperConnection Watcher:127.0.0.1:63934/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 92898 T124 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 92912 T124 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 93859 T147 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 93861 T147 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:63938_yu_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:63938_yu",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:63938/yu"}
[junit4:junit4] 2> 93869 T166 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 93869 T153 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 93869 T146 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 93917 T124 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63942_yu
[junit4:junit4] 2> 93920 T124 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:63942_yu
[junit4:junit4] 2> 93924 T153 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 93925 T146 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 93925 T166 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 93925 T146 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 93925 T166 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 93927 T153 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 93940 T167 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 93940 T167 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 95387 T147 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 95389 T147 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:63942_yu",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:63942/yu"}
[junit4:junit4] 2> 95389 T147 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4] 2> 95389 T147 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 95407 T146 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 95407 T153 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 95407 T166 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 95945 T167 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1371557354537/collection1
[junit4:junit4] 2> 95945 T167 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 95947 T167 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 95947 T167 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 95949 T167 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1371557354537/collection1/'
[junit4:junit4] 2> 95951 T167 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty1-1371557354537/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 95952 T167 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty1-1371557354537/collection1/lib/README' to classloader
[junit4:junit4] 2> 96005 T167 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 96067 T167 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 96169 T167 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 96176 T167 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 96801 T167 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 96810 T167 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 96813 T167 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 96837 T167 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 96841 T167 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 96846 T167 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 96847 T167 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 96848 T167 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 96848 T167 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 96850 T167 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 96850 T167 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 96851 T167 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 96851 T167 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1371557354537/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1371557346779/jetty1/
[junit4:junit4] 2> 96851 T167 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@790bc5c9
[junit4:junit4] 2> 96852 T167 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 96852 T167 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1371557346779/jetty1
[junit4:junit4] 2> 96853 T167 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1371557346779/jetty1/index/
[junit4:junit4] 2> 96853 T167 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1371557346779/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 96853 T167 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1371557346779/jetty1/index
[junit4:junit4] 2> 96856 T167 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@242895da lockFactory=org.apache.lucene.store.NativeFSLockFactory@653dd107),segFN=segments_1,generation=1}
[junit4:junit4] 2> 96856 T167 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 96859 T167 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 96859 T167 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 96860 T167 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 96860 T167 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 96861 T167 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 96861 T167 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 96862 T167 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 96862 T167 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 96863 T167 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 96873 T167 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 96878 T167 oass.SolrIndexSearcher.<init> Opening Searcher@616310a2 main
[junit4:junit4] 2> 96879 T167 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.RecoveryZkTest-1371557346779/jetty1/tlog
[junit4:junit4] 2> 96880 T167 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 96880 T167 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 96884 T168 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@616310a2 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 96886 T167 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 96887 T167 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:63942/yu collection:collection1 shard:shard1
[junit4:junit4] 2> 96888 T167 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 96909 T167 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 96920 T167 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 96920 T167 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 96921 T167 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:63942/yu/collection1/
[junit4:junit4] 2> 96921 T167 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 96921 T167 oasc.SyncStrategy.syncToMe http://127.0.0.1:63942/yu/collection1/ has no replicas
[junit4:junit4] 2> 96921 T167 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:63942/yu/collection1/
[junit4:junit4] 2> 96922 T167 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 98422 T147 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 98501 T146 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 98501 T166 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 98501 T153 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 98521 T167 oasc.ZkController.register We are http://127.0.0.1:63942/yu/collection1/ and leader is http://127.0.0.1:63942/yu/collection1/
[junit4:junit4] 2> 98521 T167 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:63942/yu
[junit4:junit4] 2> 98521 T167 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 98522 T167 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 98522 T167 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 98571 T167 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 98617 T124 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 98617 T124 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 98618 T124 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 98901 T124 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 98905 T124 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:63945
[junit4:junit4] 2> 98906 T124 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 98907 T124 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 98907 T124 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1371557360745
[junit4:junit4] 2> 98908 T124 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1371557360745/solr.xml
[junit4:junit4] 2> 98908 T124 oasc.CoreContainer.<init> New CoreContainer 1854895278
[junit4:junit4] 2> 98909 T124 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1371557360745/'
[junit4:junit4] 2> 98910 T124 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1371557360745/'
[junit4:junit4] 2> 99039 T124 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 99040 T124 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 99040 T124 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 99041 T124 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 99041 T124 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 99042 T124 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 99042 T124 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 99043 T124 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 99044 T124 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 99044 T124 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 99062 T124 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 99063 T124 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:63934/solr
[junit4:junit4] 2> 99064 T124 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 99065 T124 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 99071 T180 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@465652da name:ZooKeeperConnection Watcher:127.0.0.1:63934 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 99072 T124 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 99088 T124 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 99101 T124 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 99104 T182 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4f7c26b4 name:ZooKeeperConnection Watcher:127.0.0.1:63934/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 99104 T124 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 99117 T124 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 100121 T147 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 100122 T147 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:63942_yu_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:63942_yu",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:63942/yu"}
[junit4:junit4] 2> 100122 T124 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63945_yu
[junit4:junit4] 2> 100125 T124 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:63945_yu
[junit4:junit4] 2> 100141 T153 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 100142 T182 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 100143 T166 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 100143 T182 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 100143 T166 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 100145 T153 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 100145 T153 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 100153 T166 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 100154 T146 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 100154 T146 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 100156 T146 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 100159 T183 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 100160 T183 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 101657 T147 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 101659 T147 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:63945_yu",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:63945/yu"}
[junit4:junit4] 2> 101659 T147 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 101659 T147 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 101676 T153 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 101676 T166 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 101676 T146 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 101676 T182 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 102164 T183 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1371557360745/collection1
[junit4:junit4] 2> 102164 T183 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 102166 T183 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 102166 T183 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 102168 T183 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1371557360745/collection1/'
[junit4:junit4] 2> 102170 T183 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty2-1371557360745/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 102171 T183 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty2-1371557360745/collection1/lib/README' to classloader
[junit4:junit4] 2> 102235 T183 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 102299 T183 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 102401 T183 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 102408 T183 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 103085 T183 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 103101 T183 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 103105 T183 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 103133 T183 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 103139 T183 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 103145 T183 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 103147 T183 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 103148 T183 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 103148 T183 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 103150 T183 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 103150 T183 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 103151 T183 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 103151 T183 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1371557360745/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1371557346779/jetty2/
[junit4:junit4] 2> 103152 T183 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@790bc5c9
[junit4:junit4] 2> 103152 T183 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 103153 T183 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1371557346779/jetty2
[junit4:junit4] 2> 103154 T183 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1371557346779/jetty2/index/
[junit4:junit4] 2> 103154 T183 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1371557346779/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 103154 T183 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1371557346779/jetty2/index
[junit4:junit4] 2> 103159 T183 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@71f143d lockFactory=org.apache.lucene.store.NativeFSLockFactory@47edaaef),segFN=segments_1,generation=1}
[junit4:junit4] 2> 103159 T183 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 103163 T183 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 103163 T183 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 103164 T183 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 103165 T183 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 103166 T183 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 103166 T183 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 103167 T183 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 103168 T183 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 103168 T183 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 103188 T183 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 103196 T183 oass.SolrIndexSearcher.<init> Opening Searcher@5547e4d6 main
[junit4:junit4] 2> 103197 T183 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.RecoveryZkTest-1371557346779/jetty2/tlog
[junit4:junit4] 2> 103198 T183 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 103199 T183 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 103205 T184 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5547e4d6 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 103208 T183 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 103208 T183 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:63945/yu collection:collection1 shard:shard1
[junit4:junit4] 2> 103213 T183 oasc.ZkController.register We are http://127.0.0.1:63945/yu/collection1/ and leader is http://127.0.0.1:63942/yu/collection1/
[junit4:junit4] 2> 103214 T183 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:63945/yu
[junit4:junit4] 2> 103214 T183 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 103214 T183 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C18 name=collection1 org.apache.solr.core.SolrCore@e2471c7 url=http://127.0.0.1:63945/yu/collection1 node=127.0.0.1:63945_yu C18_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:63945_yu, base_url=http://127.0.0.1:63945/yu}
[junit4:junit4] 2> 103224 T185 C18 P63945 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 103224 T183 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 103224 T185 C18 P63945 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 103225 T185 C18 P63945 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 103226 T185 C18 P63945 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 103226 T124 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 103227 T124 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 103228 T124 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 103228 T185 C18 P63945 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 103255 T124 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 103286 T156 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:63945_yu_collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> ASYNC NEW_CORE C19 name=collection1 org.apache.solr.core.SolrCore@64af4b04 url=http://127.0.0.1:63938/yu/collection1 node=127.0.0.1:63938_yu C19_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:63938_yu, base_url=http://127.0.0.1:63938/yu, leader=true}
[junit4:junit4] 2> 103448 T136 C19 P63938 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@36f7bfd8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@59d76e68),segFN=segments_1,generation=1}
[junit4:junit4] 2> 103449 T136 C19 P63938 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 103456 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[1 (1438182136143151104)]} 0 79
[junit4:junit4] 2> 103456 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10001 (1438182136143151105)]} 0 79
[junit4:junit4] 2> 103460 T124 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 63945
[junit4:junit4] 2> 103460 T124 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1854895278
[junit4:junit4] 2> ASYNC NEW_CORE C20 name=collection1 org.apache.solr.core.SolrCore@186bd87 url=http://127.0.0.1:63942/yu/collection1 node=127.0.0.1:63942_yu C20_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:63942_yu, base_url=http://127.0.0.1:63942/yu, leader=true}
[junit4:junit4] 2> 103510 T157 C20 P63942 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@242895da lockFactory=org.apache.lucene.store.NativeFSLockFactory@653dd107),segFN=segments_1,generation=1}
[junit4:junit4] 2> 103511 T157 C20 P63942 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 103518 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10001 (1438182136209211392)]} 0 11
[junit4:junit4] 2> 103518 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[1 (1438182136209211393)]} 0 11
[junit4:junit4] 2> 103541 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10001 (-1438182136242765825)]} 0 1
[junit4:junit4] 2> 103541 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[1 (-1438182136242765824)]} 0 1
[junit4:junit4] 2> 103545 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10001 (-1438182136248008704)]} 0 0
[junit4:junit4] 2> 103547 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[1 (-1438182136249057280)]} 0 1
[junit4:junit4] 2> 103553 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10002 (1438182136253251584)]} 0 3
[junit4:junit4] 2> 103555 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[2 (1438182136255348736)]} 0 3
[junit4:junit4] 2> 103560 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10002 (1438182136260591616)]} 0 3
[junit4:junit4] 2> 103561 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[2 (1438182136262688768)]} 0 2
[junit4:junit4] 2> 103585 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10003 (1438182136286806016)]} 0 3
[junit4:junit4] 2> 103587 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[3 (1438182136287854592)]} 0 4
[junit4:junit4] 2> 103593 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10003 (1438182136295194624)]} 0 3
[junit4:junit4] 2> 103594 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[3 (1438182136296243200)]} 0 3
[junit4:junit4] 2> 103632 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10004 (1438182136336089088)]} 0 3
[junit4:junit4] 2> 103633 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[4 (1438182136337137664)]} 0 3
[junit4:junit4] 2> 103640 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10004 (1438182136344477696)]} 0 3
[junit4:junit4] 2> 103641 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[4 (1438182136345526272)]} 0 3
[junit4:junit4] 2> 103741 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10002 (-1438182136452481024)]} 0 1
[junit4:junit4] 2> 103741 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[2 (-1438182136452481025)]} 0 1
[junit4:junit4] 2> 103745 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10002 (-1438182136457723904)]} 0 0
[junit4:junit4] 2> 103746 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[2 (-1438182136458772480)]} 0 0
[junit4:junit4] 2> 103753 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[5 (1438182136462966785)]} 0 3
[junit4:junit4] 2> 103753 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10005 (1438182136462966784)]} 0 3
[junit4:junit4] 2> 103761 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[5 (1438182136471355393)]} 0 3
[junit4:junit4] 2> 103761 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10005 (1438182136471355392)]} 0 3
[junit4:junit4] 2> 103776 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[3 (-1438182136490229760)]} 0 0
[junit4:junit4] 2> 103777 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10003 (-1438182136491278336)]} 0 0
[junit4:junit4] 2> 103781 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10003 (-1438182136495472640)]} 0 0
[junit4:junit4] 2> 103782 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[3 (-1438182136496521216)]} 0 1
[junit4:junit4] 2> 103789 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[6 (1438182136500715521)]} 0 3
[junit4:junit4] 2> 103789 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10006 (1438182136500715520)]} 0 3
[junit4:junit4] 2> 103797 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10006 (1438182136509104128)]} 0 3
[junit4:junit4] 2> 103797 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[6 (1438182136509104129)]} 0 3
[junit4:junit4] 2> 103895 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10005 (-1438182136615010304)]} 0 1
[junit4:junit4] 2> 103895 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[5 (-1438182136615010305)]} 0 0
[junit4:junit4] 2> 103899 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10005 (-1438182136619204608)]} 0 0
[junit4:junit4] 2> 103900 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[5 (-1438182136620253184)]} 0 0
[junit4:junit4] 2> 103906 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10007 (1438182136623398912)]} 0 3
[junit4:junit4] 2> 103908 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[7 (1438182136625496064)]} 0 3
[junit4:junit4] 2> 103914 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10007 (1438182136631787520)]} 0 3
[junit4:junit4] 2> 103916 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[7 (1438182136633884672)]} 0 3
[junit4:junit4] 2> 104009 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10008 (1438182136731402240)]} 0 4
[junit4:junit4] 2> 104010 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[8 (1438182136732450816)]} 0 2
[junit4:junit4] 2> 104016 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10008 (1438182136738742272)]} 0 3
[junit4:junit4] 2> 104017 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[8 (1438182136739790848)]} 0 3
[junit4:junit4] 2> 104101 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[9 (1438182136827871232)]} 0 3
[junit4:junit4] 2> 104101 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10009 (1438182136826822656)]} 0 4
[junit4:junit4] 2> 104108 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[9 (1438182136835211264)]} 0 3
[junit4:junit4] 2> 104108 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10009 (1438182136836259840)]} 0 2
[junit4:junit4] 2> 104149 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[9 (-1438182136880300032)]} 0 1
[junit4:junit4] 2> 104150 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10009 (-1438182136881348608)]} 0 0
[junit4:junit4] 2> 104154 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10009 (-1438182136886591488)]} 0 0
[junit4:junit4] 2> 104155 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[9 (-1438182136887640064)]} 0 1
[junit4:junit4] 2> 104162 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10010 (1438182136891834368)]} 0 3
[junit4:junit4] 2> 104162 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10 (1438182136892882944)]} 0 3
[junit4:junit4] 2> 104169 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10010 (1438182136899174400)]} 0 3
[junit4:junit4] 2> 104170 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10 (1438182136900222976)]} 0 3
[junit4:junit4] 2> 104241 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10011 (1438182136974671872)]} 0 3
[junit4:junit4] 2> 104241 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[11 (1438182136975720448)]} 0 2
[junit4:junit4] 2> 104247 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10011 (1438182136982011904)]} 0 3
[junit4:junit4] 2> 104249 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[11 (1438182136983060480)]} 0 3
[junit4:junit4] 2> 104330 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10012 (1438182137069043712)]} 0 3
[junit4:junit4] 2> 104332 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[12 (1438182137070092288)]} 0 3
[junit4:junit4] 2> 104338 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10012 (1438182137076383744)]} 0 3
[junit4:junit4] 2> 104339 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[12 (1438182137077432320)]} 0 3
[junit4:junit4] 2> 104415 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10010 (-1438182137159221248)]} 0 1
[junit4:junit4] 2> 104416 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10 (-1438182137160269824)]} 0 1
[junit4:junit4] 2> 104419 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10010 (-1438182137164464128)]} 0 0
[junit4:junit4] 2> 104421 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10 (-1438182137165512704)]} 0 1
[junit4:junit4] 2> 104427 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10013 (1438182137169707008)]} 0 3
[junit4:junit4] 2> 104428 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[13 (1438182137170755584)]} 0 3
[junit4:junit4] 2> 104435 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[13 (1438182137178095617)]} 0 3
[junit4:junit4] 2> 104435 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10013 (1438182137178095616)]} 0 3
[junit4:junit4] 2> 104462 T124 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 104463 T124 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 104466 T124 oasc.RecoveryStrategy.close WARN Stopping recovery for zkNodeName=127.0.0.1:63945_yu_collection1core=collection1
[junit4:junit4] 2> 104512 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10014 (1438182137258835968)]} 0 4
[junit4:junit4] 2> 104512 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[14 (1438182137258835969)]} 0 4
[junit4:junit4] 2> 104518 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10014 (1438182137266176000)]} 0 2
[junit4:junit4] 2> 104520 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[14 (1438182137267224576)]} 0 3
[junit4:junit4] 2> 104597 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10012 (-1438182137351110656)]} 0 0
[junit4:junit4] 2> 104599 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[12 (-1438182137353207808)]} 0 0
[junit4:junit4] 2> 104602 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10012 (-1438182137356353536)]} 0 0
[junit4:junit4] 2> 104603 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[12 (-1438182137357402112)]} 0 0
[junit4:junit4] 2> 104609 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10015 (1438182137360547840)]} 0 3
[junit4:junit4] 2> 104610 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[15 (1438182137362644992)]} 0 3
[junit4:junit4] 2> 104617 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10015 (1438182137368936448)]} 0 3
[junit4:junit4] 2> 104618 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[15 (1438182137369985024)]} 0 4
[junit4:junit4] 2> 104632 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10016 (1438182137385713664)]} 0 3
[junit4:junit4] 2> 104633 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[16 (1438182137386762240)]} 0 3
[junit4:junit4] 2> 104639 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[16 (1438182137393053696)]} 0 3
[junit4:junit4] 2> 104640 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10016 (1438182137393053697)]} 0 3
[junit4:junit4] 2> 104687 T147 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 104688 T147 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:63945_yu_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:63945_yu",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:63945/yu"}
[junit4:junit4] 2> 104711 T147 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:63945_yu_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:63945_yu",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:63945/yu"}
[junit4:junit4] 2> 104723 T153 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 104723 T166 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 104723 T146 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 104723 T182 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 104737 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[17 (1438182137494765568)]} 0 3
[junit4:junit4] 2> 104737 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10017 (1438182137495814144)]} 0 2
[junit4:junit4] 2> 104743 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[17 (1438182137502105600)]} 0 3
[junit4:junit4] 2> 104744 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10017 (1438182137503154176)]} 0 3
[junit4:junit4] 2> 104747 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[15 (-1438182137508397056)]} 0 1
[junit4:junit4] 2> 104748 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10015 (-1438182137509445632)]} 0 1
[junit4:junit4] 2> 104751 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[15 (-1438182137512591360)]} 0 1
[junit4:junit4] 2> 104753 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10015 (-1438182137513639936)]} 0 0
[junit4:junit4] 2> 104758 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[18 (1438182137517834240)]} 0 3
[junit4:junit4] 2> 104759 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10018 (1438182137518882816)]} 0 3
[junit4:junit4] 2> 104764 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[18 (1438182137524125696)]} 0 3
[junit4:junit4] 2> 104765 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10018 (1438182137525174272)]} 0 3
[junit4:junit4] 2> 104831 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[16 (-1438182137596477440)]} 0 0
[junit4:junit4] 2> 104832 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10016 (-1438182137597526016)]} 0 1
[junit4:junit4] 2> 104836 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[16 (-1438182137601720321)]} 0 0
[junit4:junit4] 2> 104836 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10016 (-1438182137601720320)]} 0 0
[junit4:junit4] 2> 104841 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[19 (1438182137604866048)]} 0 3
[junit4:junit4] 2> 104843 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10019 (1438182137606963200)]} 0 2
[junit4:junit4] 2> 104847 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[19 (1438182137611157504)]} 0 3
[junit4:junit4] 2> 104849 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10019 (1438182137613254656)]} 0 2
[junit4:junit4] 2> 104853 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[17 (-1438182137619546112)]} 0 1
[junit4:junit4] 2> 104855 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10017 (-1438182137621643264)]} 0 1
[junit4:junit4] 2> 104857 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[17 (-1438182137623740416)]} 0 1
[junit4:junit4] 2> 104859 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10017 (-1438182137625837568)]} 0 1
[junit4:junit4] 2> 104863 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[20 (1438182137627934720)]} 0 3
[junit4:junit4] 2> 104865 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10020 (1438182137630031872)]} 0 3
[junit4:junit4] 2> 104869 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[20 (1438182137634226176)]} 0 3
[junit4:junit4] 2> 104871 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10020 (1438182137636323328)]} 0 3
[junit4:junit4] 2> 104956 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[21 (1438182137725452288)]} 0 2
[junit4:junit4] 2> 104958 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10021 (1438182137727549440)]} 0 3
[junit4:junit4] 2> 104963 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[21 (1438182137732792320)]} 0 3
[junit4:junit4] 2> 104964 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10021 (1438182137733840896)]} 0 3
[junit4:junit4] 2> 105029 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[18 (-1438182137804095488)]} 0 0
[junit4:junit4] 2> 105030 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10018 (-1438182137805144064)]} 0 1
[junit4:junit4] 2> 105034 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[18 (-1438182137809338369)]} 0 0
[junit4:junit4] 2> 105034 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10018 (-1438182137809338368)]} 0 0
[junit4:junit4] 2> 105041 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[22 (1438182137814581248)]} 0 3
[junit4:junit4] 2> 105041 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10022 (1438182137814581249)]} 0 3
[junit4:junit4] 2> 105048 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[22 (1438182137821921280)]} 0 3
[junit4:junit4] 2> 105048 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10022 (1438182137821921281)]} 0 2
[junit4:junit4] 2> 105085 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10023 (1438182137859670017)]} 0 3
[junit4:junit4] 2> 105085 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[23 (1438182137859670016)]} 0 3
[junit4:junit4] 2> 105091 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10023 (1438182137867010048)]} 0 3
[junit4:junit4] 2> 105092 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[23 (1438182137868058624)]} 0 3
[junit4:junit4] 2> 105179 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10024 (1438182137959284736)]} 0 2
[junit4:junit4] 2> 105180 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[24 (1438182137960333312)]} 0 2
[junit4:junit4] 2> 105186 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10024 (1438182137966624768)]} 0 3
[junit4:junit4] 2> 105187 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[24 (1438182137967673344)]} 0 3
[junit4:junit4] 2> 105265 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10025 (1438182138049462272)]} 0 2
[junit4:junit4] 2> 105266 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[25 (1438182138050510848)]} 0 2
[junit4:junit4] 2> 105271 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10025 (1438182138055753728)]} 0 3
[junit4:junit4] 2> 105272 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[25 (1438182138056802304)]} 0 2
[junit4:junit4] 2> 105347 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10026 (1438182138135445504)]} 0 2
[junit4:junit4] 2> 105348 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[26 (1438182138136494080)]} 0 3
[junit4:junit4] 2> 105354 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[26 (1438182138142785536)]} 0 3
[junit4:junit4] 2> 105354 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10026 (1438182138142785537)]} 0 3
[junit4:junit4] 2> 105423 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[27 (1438182138214088704)]} 0 3
[junit4:junit4] 2> 105423 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10027 (1438182138215137280)]} 0 3
[junit4:junit4] 2> 105429 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[27 (1438182138221428736)]} 0 3
[junit4:junit4] 2> 105430 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10027 (1438182138222477312)]} 0 2
[junit4:junit4] 2> 105506 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[28 (1438182138301120512)]} 0 3
[junit4:junit4] 2> 105506 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10028 (1438182138301120513)]} 0 3
[junit4:junit4] 2> 105513 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[28 (1438182138309509120)]} 0 2
[junit4:junit4] 2> 105514 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10028 (1438182138310557696)]} 0 3
[junit4:junit4] 2> 105591 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[29 (1438182138391298048)]} 0 2
[junit4:junit4] 2> 105592 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10029 (1438182138392346624)]} 0 3
[junit4:junit4] 2> 105598 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10029 (1438182138398638080)]} 0 3
[junit4:junit4] 2> 105598 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[29 (1438182138398638081)]} 0 2
[junit4:junit4] 2> 105631 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[20 (-1438182138435338240)]} 0 1
[junit4:junit4] 2> 105631 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10020 (-1438182138435338241)]} 0 1
[junit4:junit4] 2> 105635 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[20 (-1438182138439532544)]} 0 1
[junit4:junit4] 2> 105636 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10020 (-1438182138440581120)]} 0 0
[junit4:junit4] 2> 105641 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[30 (1438182138443726848)]} 0 3
[junit4:junit4] 2> 105642 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10030 (1438182138444775424)]} 0 3
[junit4:junit4] 2> 105647 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[30 (1438182138450018304)]} 0 3
[junit4:junit4] 2> 105648 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10030 (1438182138451066880)]} 0 3
[junit4:junit4] 2> 105739 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[26 (-1438182138547535872)]} 0 1
[junit4:junit4] 2> 105739 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10026 (-1438182138548584448)]} 0 0
[junit4:junit4] 2> 105743 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[26 (-1438182138552778752)]} 0 1
[junit4:junit4] 2> 105744 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10026 (-1438182138553827328)]} 0 1
[junit4:junit4] 2> 105749 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[31 (1438182138556973056)]} 0 3
[junit4:junit4] 2> 105750 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10031 (1438182138558021632)]} 0 3
[junit4:junit4] 2> 105755 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[31 (1438182138563264512)]} 0 3
[junit4:junit4] 2> 105756 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10031 (1438182138564313088)]} 0 3
[junit4:junit4] 2> 105857 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[27 (-1438182138672316417)]} 0 0
[junit4:junit4] 2> 105857 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10027 (-1438182138672316416)]} 0 0
[junit4:junit4] 2> 105862 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[27 (-1438182138677559296)]} 0 1
[junit4:junit4] 2> 105863 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10027 (-1438182138678607872)]} 0 1
[junit4:junit4] 2> 105868 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[32 (1438182138681753600)]} 0 3
[junit4:junit4] 2> 105869 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10032 (1438182138682802176)]} 0 3
[junit4:junit4] 2> 105874 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[32 (1438182138688045056)]} 0 3
[junit4:junit4] 2> 105875 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10032 (1438182138689093632)]} 0 3
[junit4:junit4] 2> 105958 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[28 (-1438182138777174016)]} 0 1
[junit4:junit4] 2> 105958 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10028 (-1438182138778222592)]} 0 0
[junit4:junit4] 2> 105962 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[28 (-1438182138782416896)]} 0 1
[junit4:junit4] 2> 105963 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10028 (-1438182138783465472)]} 0 1
[junit4:junit4] 2> 105968 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[33 (1438182138786611200)]} 0 3
[junit4:junit4] 2> 105970 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10033 (1438182138788708352)]} 0 3
[junit4:junit4] 2> 105974 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[33 (1438182138792902656)]} 0 3
[junit4:junit4] 2> 105976 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10033 (1438182138794999808)]} 0 3
[junit4:junit4] 2> 106068 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[29 (-1438182138893565952)]} 0 0
[junit4:junit4] 2> 106070 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10029 (-1438182138895663104)]} 0 0
[junit4:junit4] 2> 106073 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[29 (-1438182138898808832)]} 0 1
[junit4:junit4] 2> 106075 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10029 (-1438182138900905984)]} 0 1
[junit4:junit4] 2> 106079 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[34 (1438182138903003136)]} 0 3
[junit4:junit4] 2> 106081 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10034 (1438182138905100288)]} 0 3
[junit4:junit4] 2> 106085 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[34 (1438182138909294592)]} 0 3
[junit4:junit4] 2> 106086 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10034 (1438182138911391744)]} 0 2
[junit4:junit4] 2> 106173 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[35 (1438182139001569280)]} 0 2
[junit4:junit4] 2> 106175 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10035 (1438182139003666432)]} 0 3
[junit4:junit4] 2> 106180 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[35 (1438182139008909312)]} 0 3
[junit4:junit4] 2> 106181 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10035 (1438182139009957888)]} 0 2
[junit4:junit4] 2> 106239 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[30 (-1438182139072872448)]} 0 0
[junit4:junit4] 2> 106240 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10030 (-1438182139073921024)]} 0 1
[junit4:junit4] 2> 106243 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[30 (-1438182139077066752)]} 0 1
[junit4:junit4] 2> 106245 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10030 (-1438182139079163904)]} 0 1
[junit4:junit4] 2> 106249 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[36 (1438182139081261056)]} 0 3
[junit4:junit4] 2> 106250 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10036 (1438182139083358208)]} 0 2
[junit4:junit4] 2> 106254 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[36 (1438182139087552512)]} 0 2
[junit4:junit4] 2> 106257 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10036 (1438182139089649664)]} 0 3
[junit4:junit4] 2> 106335 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[32 (-1438182139172487168)]} 0 1
[junit4:junit4] 2> 106336 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10032 (-1438182139174584320)]} 0 1
[junit4:junit4] 2> 106339 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[32 (-1438182139177730048)]} 0 0
[junit4:junit4] 2> 106340 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10032 (-1438182139178778624)]} 0 1
[junit4:junit4] 2> 106346 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[37 (1438182139182972928)]} 0 3
[junit4:junit4] 2> 106346 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10037 (1438182139182972929)]} 0 3
[junit4:junit4] 2> 106352 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[37 (1438182139189264384)]} 0 3
[junit4:junit4] 2> 106353 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10037 (1438182139190312960)]} 0 3
[junit4:junit4] 2> 106455 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[38 (1438182139297267712)]} 0 2
[junit4:junit4] 2> 106456 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10038 (1438182139298316288)]} 0 3
[junit4:junit4] 2> 106462 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[38 (1438182139304607744)]} 0 2
[junit4:junit4] 2> 106463 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10038 (1438182139305656320)]} 0 3
[junit4:junit4] 2> 106487 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[39 (1438182139330822144)]} 0 3
[junit4:junit4] 2> 106487 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10039 (1438182139331870720)]} 0 2
[junit4:junit4] 2> 106492 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[39 (1438182139337113600)]} 0 2
[junit4:junit4] 2> 106493 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10039 (1438182139338162176)]} 0 2
[junit4:junit4] 2> 106532 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[33 (-1438182139380105216)]} 0 1
[junit4:junit4] 2> 106533 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10033 (-1438182139381153792)]} 0 1
[junit4:junit4] 2> 106536 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[33 (-1438182139384299520)]} 0 1
[junit4:junit4] 2> 106537 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10033 (-1438182139385348096)]} 0 1
[junit4:junit4] 2> 106542 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[40 (1438182139388493824)]} 0 3
[junit4:junit4] 2> 106543 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10040 (1438182139389542400)]} 0 2
[junit4:junit4] 2> 106548 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[40 (1438182139394785280)]} 0 3
[junit4:junit4] 2> 106548 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10040 (1438182139395833856)]} 0 2
[junit4:junit4] 2> 106638 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10035 (-1438182139491254273)]} 0 0
[junit4:junit4] 2> 106638 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[35 (-1438182139491254272)]} 0 0
[junit4:junit4] 2> 106643 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10035 (-1438182139496497152)]} 0 1
[junit4:junit4] 2> 106643 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[35 (-1438182139496497153)]} 0 1
[junit4:junit4] 2> 106649 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10041 (1438182139500691456)]} 0 3
[junit4:junit4] 2> 106649 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[41 (1438182139501740032)]} 0 2
[junit4:junit4] 2> 106654 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10041 (1438182139506982912)]} 0 2
[junit4:junit4] 2> 106655 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[41 (1438182139508031488)]} 0 2
[junit4:junit4] 2> 106755 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10042 (1438182139611840512)]} 0 2
[junit4:junit4] 2> 106756 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[42 (1438182139612889088)]} 0 3
[junit4:junit4] 2> 106761 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[42 (1438182139619180544)]} 0 2
[junit4:junit4] 2> 106761 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10042 (1438182139619180545)]} 0 2
[junit4:junit4] 2> 106795 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[43 (1438182139653783552)]} 0 3
[junit4:junit4] 2> 106796 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10043 (1438182139654832128)]} 0 2
[junit4:junit4] 2> 106800 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[43 (1438182139660075008)]} 0 2
[junit4:junit4] 2> 106801 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10043 (1438182139661123584)]} 0 2
[junit4:junit4] 2> 106875 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[44 (1438182139737669632)]} 0 2
[junit4:junit4] 2> 106875 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10044 (1438182139738718208)]} 0 2
[junit4:junit4] 2> 106881 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[44 (1438182139745009664)]} 0 2
[junit4:junit4] 2> 106881 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10044 (1438182139745009665)]} 0 2
[junit4:junit4] 2> 106910 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10037 (-1438182139776466944)]} 0 1
[junit4:junit4] 2> 106910 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[37 (-1438182139776466945)]} 0 1
[junit4:junit4] 2> 106914 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10037 (-1438182139780661248)]} 0 1
[junit4:junit4] 2> 106914 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[37 (-1438182139780661249)]} 0 1
[junit4:junit4] 2> 106919 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10045 (1438182139784855552)]} 0 2
[junit4:junit4] 2> 106920 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[45 (1438182139785904128)]} 0 1
[junit4:junit4] 2> 106924 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10045 (1438182139790098432)]} 0 2
[junit4:junit4] 2> 106926 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[45 (1438182139792195584)]} 0 2
[junit4:junit4] 2> 106939 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10046 (1438182139805827072)]} 0 2
[junit4:junit4] 2> 106941 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[46 (1438182139807924224)]} 0 2
[junit4:junit4] 2> 106946 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10046 (1438182139812118528)]} 0 3
[junit4:junit4] 2> 106946 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[46 (1438182139813167104)]} 0 2
[junit4:junit4] 2> 106972 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[47 (1438182139840430080)]} 0 2
[junit4:junit4] 2> 106972 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10047 (1438182139840430081)]} 0 2
[junit4:junit4] 2> 106977 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[47 (1438182139845672960)]} 0 2
[junit4:junit4] 2> 106979 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10047 (1438182139846721536)]} 0 3
[junit4:junit4] 2> 106986 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[48 (1438182139855110144)]} 0 1
[junit4:junit4] 2> 106988 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10048 (1438182139857207296)]} 0 2
[junit4:junit4] 2> 106993 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[48 (1438182139861401600)]} 0 3
[junit4:junit4] 2> 106993 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10048 (1438182139862450176)]} 0 2
[junit4:junit4] 2> 107030 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10049 (1438182139901247488)]} 0 2
[junit4:junit4] 2> 107031 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[49 (1438182139901247489)]} 0 2
[junit4:junit4] 2> 107036 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10049 (1438182139907538944)]} 0 2
[junit4:junit4] 2> 107037 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[49 (1438182139907538945)]} 0 2
[junit4:junit4] 2> 107100 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10038 (-1438182139975696384)]} 0 0
[junit4:junit4] 2> 107101 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[38 (-1438182139976744960)]} 0 1
[junit4:junit4] 2> 107105 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10038 (-1438182139980939265)]} 0 1
[junit4:junit4] 2> 107105 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[38 (-1438182139980939264)]} 0 1
[junit4:junit4] 2> 107110 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10050 (1438182139985133568)]} 0 2
[junit4:junit4] 2> 107111 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[50 (1438182139986182144)]} 0 2
[junit4:junit4] 2> 107116 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[50 (1438182139991425024)]} 0 2
[junit4:junit4] 2> 107116 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10050 (1438182139991425025)]} 0 2
[junit4:junit4] 2> 107182 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[51 (1438182140060631040)]} 0 2
[junit4:junit4] 2> 107183 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10051 (1438182140061679616)]} 0 1
[junit4:junit4] 2> 107187 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[51 (1438182140065873920)]} 0 2
[junit4:junit4] 2> 107189 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10051 (1438182140067971072)]} 0 2
[junit4:junit4] 2> 107199 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[52 (1438182140078456832)]} 0 2
[junit4:junit4] 2> 107201 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10052 (1438182140080553984)]} 0 2
[junit4:junit4] 2> 107206 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[52 (1438182140084748288)]} 0 2
[junit4:junit4] 2> 107206 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10052 (1438182140085796864)]} 0 2
[junit4:junit4] 2> 107233 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10053 (1438182140114108416)]} 0 2
[junit4:junit4] 2> 107233 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[53 (1438182140114108417)]} 0 2
[junit4:junit4] 2> 107239 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10053 (1438182140120399873)]} 0 1
[junit4:junit4] 2> 107239 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[53 (1438182140120399872)]} 0 1
[junit4:junit4] 2> 107294 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10040 (-1438182140179120128)]} 0 1
[junit4:junit4] 2> 107295 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[40 (-1438182140179120129)]} 0 1
[junit4:junit4] 2> 107298 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10040 (-1438182140182265856)]} 0 1
[junit4:junit4] 2> 107299 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[40 (-1438182140184363008)]} 0 0
[junit4:junit4] 2> 107303 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10054 (1438182140187508736)]} 0 2
[junit4:junit4] 2> 107304 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[54 (1438182140188557312)]} 0 2
[junit4:junit4] 2> 107308 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10054 (1438182140192751616)]} 0 2
[junit4:junit4] 2> 107309 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[54 (1438182140193800192)]} 0 2
[junit4:junit4] 2> 107358 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10055 (1438182140244131840)]} 0 2
[junit4:junit4] 2> 107359 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[55 (1438182140246228992)]} 0 2
[junit4:junit4] 2> 107363 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10055 (1438182140250423296)]} 0 2
[junit4:junit4] 2> 107364 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[55 (1438182140251471872)]} 0 2
[junit4:junit4] 2> 107425 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10056 (1438182140314386433)]} 0 2
[junit4:junit4] 2> 107425 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[56 (1438182140314386432)]} 0 2
[junit4:junit4] 2> 107431 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[56 (1438182140321726464)]} 0 2
[junit4:junit4] 2> 107431 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10056 (1438182140321726465)]} 0 2
[junit4:junit4] 2> 107466 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[41 (-1438182140359475200)]} 0 0
[junit4:junit4] 2> 107467 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10041 (-1438182140360523776)]} 0 1
[junit4:junit4] 2> 107471 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10041 (-1438182140364718080)]} 0 0
[junit4:junit4] 2> 107471 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[41 (-1438182140364718081)]} 0 0
[junit4:junit4] 2> 107476 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10057 (1438182140368912384)]} 0 2
[junit4:junit4] 2> 107477 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[57 (1438182140369960960)]} 0 2
[junit4:junit4] 2> 107482 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10057 (1438182140375203840)]} 0 2
[junit4:junit4] 2> 107482 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[57 (1438182140375203841)]} 0 2
[junit4:junit4] 2> 107533 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10042 (-1438182140429729792)]} 0 1
[junit4:junit4] 2> 107534 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[42 (-1438182140430778368)]} 0 1
[junit4:junit4] 2> 107537 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10042 (-1438182140433924096)]} 0 1
[junit4:junit4] 2> 107538 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[42 (-1438182140434972672)]} 0 1
[junit4:junit4] 2> 107542 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10058 (1438182140438118400)]} 0 2
[junit4:junit4] 2> 107544 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[58 (1438182140440215552)]} 0 2
[junit4:junit4] 2> 107547 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10058 (1438182140443361280)]} 0 2
[junit4:junit4] 2> 107550 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[58 (1438182140446507008)]} 0 2
[junit4:junit4] 2> 107556 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10059 (1438182140452798464)]} 0 2
[junit4:junit4] 2> 107559 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[59 (1438182140455944192)]} 0 2
[junit4:junit4] 2> 107562 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10059 (1438182140459089920)]} 0 2
[junit4:junit4] 2> 107565 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[59 (1438182140462235648)]} 0 2
[junit4:junit4] 2> 107585 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10044 (-1438182140484255744)]} 0 1
[junit4:junit4] 2> 107588 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[44 (-1438182140486352896)]} 0 1
[junit4:junit4] 2> 107589 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10044 (-1438182140488450048)]} 0 1
[junit4:junit4] 2> 107592 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[44 (-1438182140491595776)]} 0 1
[junit4:junit4] 2> 107594 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10060 (1438182140492644352)]} 0 2
[junit4:junit4] 2> 107597 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[60 (1438182140495790080)]} 0 2
[junit4:junit4] 2> 107599 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10060 (1438182140497887232)]} 0 2
[junit4:junit4] 2> 107602 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[60 (1438182140501032960)]} 0 2
[junit4:junit4] 2> 107691 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10061 (1438182140594356224)]} 0 2
[junit4:junit4] 2> 107694 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[61 (1438182140597501952)]} 0 2
[junit4:junit4] 2> 107697 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10061 (1438182140600647680)]} 0 2
[junit4:junit4] 2> 107699 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[61 (1438182140602744832)]} 0 2
[junit4:junit4] 2> 107773 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10045 (-1438182140681388032)]} 0 1
[junit4:junit4] 2> 107775 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[45 (-1438182140683485184)]} 0 1
[junit4:junit4] 2> 107777 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10045 (-1438182140684533760)]} 0 1
[junit4:junit4] 2> 107779 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[45 (-1438182140686630912)]} 0 1
[junit4:junit4] 2> 107782 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10062 (1438182140689776640)]} 0 2
[junit4:junit4] 2> 107784 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[62 (1438182140691873792)]} 0 2
[junit4:junit4] 2> 107788 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10062 (1438182140696068096)]} 0 2
[junit4:junit4] 2> 107789 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[62 (1438182140697116672)]} 0 2
[junit4:junit4] 2> 107851 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10047 (-1438182140763176960)]} 0 1
[junit4:junit4] 2> 107852 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[47 (-1438182140763176961)]} 0 1
[junit4:junit4] 2> 107855 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10047 (-1438182140766322688)]} 0 1
[junit4:junit4] 2> 107856 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[47 (-1438182140768419840)]} 0 1
[junit4:junit4] 2> 107860 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10063 (1438182140771565568)]} 0 2
[junit4:junit4] 2> 107861 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[63 (1438182140772614144)]} 0 2
[junit4:junit4] 2> 107865 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10063 (1438182140776808448)]} 0 2
[junit4:junit4] 2> 107866 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[63 (1438182140777857024)]} 0 2
[junit4:junit4] 2> 107890 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10049 (-1438182140804071424)]} 0 1
[junit4:junit4] 2> 107891 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[49 (-1438182140804071425)]} 0 1
[junit4:junit4] 2> 107894 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10049 (-1438182140807217152)]} 0 1
[junit4:junit4] 2> 107895 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[49 (-1438182140808265728)]} 0 1
[junit4:junit4] 2> 107899 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10064 (1438182140812460032)]} 0 2
[junit4:junit4] 2> 107900 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[64 (1438182140813508608)]} 0 2
[junit4:junit4] 2> 107904 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10064 (1438182140817702912)]} 0 2
[junit4:junit4] 2> 107906 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[64 (1438182140819800064)]} 0 2
[junit4:junit4] 2> 107934 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10051 (-1438182140850208768)]} 0 1
[junit4:junit4] 2> 107936 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[51 (-1438182140852305920)]} 0 1
[junit4:junit4] 2> 107938 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10051 (-1438182140853354496)]} 0 1
[junit4:junit4] 2> 107940 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[51 (-1438182140856500224)]} 0 1
[junit4:junit4] 2> 107943 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10065 (1438182140858597376)]} 0 2
[junit4:junit4] 2> 107945 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[65 (1438182140860694528)]} 0 2
[junit4:junit4] 2> 107948 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10065 (1438182140863840256)]} 0 2
[junit4:junit4] 2> 107950 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[65 (1438182140865937408)]} 0 2
[junit4:junit4] 2> 108015 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10055 (-1438182140935143424)]} 0 0
[junit4:junit4] 2> 108017 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[55 (-1438182140937240576)]} 0 1
[junit4:junit4] 2> 108020 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10055 (-1438182140939337728)]} 0 1
[junit4:junit4] 2> 108021 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[55 (-1438182140941434880)]} 0 1
[junit4:junit4] 2> 108025 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10066 (1438182140944580608)]} 0 2
[junit4:junit4] 2> 108026 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[66 (1438182140945629184)]} 0 2
[junit4:junit4] 2> 108031 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[66 (1438182140950872065)]} 0 2
[junit4:junit4] 2> 108031 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10066 (1438182140950872064)]} 0 2
[junit4:junit4] 2> 108105 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[56 (-1438182141029515264)]} 0 1
[junit4:junit4] 2> 108106 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10056 (-1438182141029515265)]} 0 1
[junit4:junit4] 2> 108110 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10056 (-1438182141033709568)]} 0 1
[junit4:junit4] 2> 108110 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[56 (-1438182141034758144)]} 0 1
[junit4:junit4] 2> 108115 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[67 (1438182141038952448)]} 0 2
[junit4:junit4] 2> 108116 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10067 (1438182141038952449)]} 0 2
[junit4:junit4] 2> 108120 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[67 (1438182141044195328)]} 0 2
[junit4:junit4] 2> 108121 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10067 (1438182141045243904)]} 0 2
[junit4:junit4] 2> 108160 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[68 (1438182141085089792)]} 0 3
[junit4:junit4] 2> 108160 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10068 (1438182141086138368)]} 0 2
[junit4:junit4] 2> 108165 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[68 (1438182141091381248)]} 0 2
[junit4:junit4] 2> 108166 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10068 (1438182141092429824)]} 0 2
[junit4:junit4] 2> 108205 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[69 (1438182141133324288)]} 0 2
[junit4:junit4] 2> 108206 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10069 (1438182141134372864)]} 0 2
[junit4:junit4] 2> 108210 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[69 (1438182141138567168)]} 0 2
[junit4:junit4] 2> 108212 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10069 (1438182141140664320)]} 0 2
[junit4:junit4] 2> 108270 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[57 (-1438182141202530304)]} 0 0
[junit4:junit4] 2> 108272 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10057 (-1438182141204627456)]} 0 1
[junit4:junit4] 2> 108275 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[57 (-1438182141207773184)]} 0 1
[junit4:junit4] 2> 108276 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10057 (-1438182141207773185)]} 0 1
[junit4:junit4] 2> 108280 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[70 (1438182141211967488)]} 0 2
[junit4:junit4] 2> 108281 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10070 (1438182141213016064)]} 0 2
[junit4:junit4] 2> 108285 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[70 (1438182141217210368)]} 0 2
[junit4:junit4] 2> 108286 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10070 (1438182141218258944)]} 0 2
[junit4:junit4] 2> 108342 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[71 (1438182141275930624)]} 0 2
[junit4:junit4] 2> 108342 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10071 (1438182141276979200)]} 0 2
[junit4:junit4] 2> 108347 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[71 (1438182141282222080)]} 0 2
[junit4:junit4] 2> 108348 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10071 (1438182141283270656)]} 0 2
[junit4:junit4] 2> 108451 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[58 (-1438182141392322560)]} 0 0
[junit4:junit4] 2> 108452 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10058 (-1438182141393371136)]} 0 1
[junit4:junit4] 2> 108456 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[58 (-1438182141397565440)]} 0 1
[junit4:junit4] 2> 108456 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10058 (-1438182141397565441)]} 0 1
[junit4:junit4] 2> 108461 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[72 (1438182141401759744)]} 0 2
[junit4:junit4] 2> 108462 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10072 (1438182141402808320)]} 0 2
[junit4:junit4] 2> 108467 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[72 (1438182141408051200)]} 0 2
[junit4:junit4] 2> 108467 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10072 (1438182141408051201)]} 0 2
[junit4:junit4] 2> 108515 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[60 (-1438182141459431424)]} 0 1
[junit4:junit4] 2> 108516 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10060 (-1438182141460480000)]} 0 1
[junit4:junit4] 2> 108519 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[60 (-1438182141463625728)]} 0 1
[junit4:junit4] 2> 108520 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10060 (-1438182141463625729)]} 0 1
[junit4:junit4] 2> 108524 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[73 (1438182141467820032)]} 0 2
[junit4:junit4] 2> 108525 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10073 (1438182141468868608)]} 0 2
[junit4:junit4] 2> 108529 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[73 (1438182141473062912)]} 0 2
[junit4:junit4] 2> 108530 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10073 (1438182141474111488)]} 0 2
[junit4:junit4] 2> 108616 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[74 (1438182141564289024)]} 0 2
[junit4:junit4] 2> 108617 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10074 (1438182141565337600)]} 0 2
[junit4:junit4] 2> 108621 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[74 (1438182141569531904)]} 0 2
[junit4:junit4] 2> 108622 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10074 (1438182141570580480)]} 0 2
[junit4:junit4] 2> 108726 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[75 (1438182141679632384)]} 0 2
[junit4:junit4] 2> 108727 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10075 (1438182141680680960)]} 0 2
[junit4:junit4] 2> 108732 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10075 (1438182141685923841)]} 0 2
[junit4:junit4] 2> 108732 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[75 (1438182141685923840)]} 0 2
[junit4:junit4] 2> 108756 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10061 (-1438182141712138241)]} 0 1
[junit4:junit4] 2> 108756 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[61 (-1438182141712138240)]} 0 1
[junit4:junit4] 2> 108761 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[61 (-1438182141717381120)]} 0 1
[junit4:junit4] 2> 108761 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10061 (-1438182141717381121)]} 0 1
[junit4:junit4] 2> 108766 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[76 (1438182141721575424)]} 0 2
[junit4:junit4] 2> 108767 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10076 (1438182141722624000)]} 0 2
[junit4:junit4] 2> 108771 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[76 (1438182141726818304)]} 0 2
[junit4:junit4] 2> 108772 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10076 (1438182141727866880)]} 0 2
[junit4:junit4] 2> 108792 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[77 (1438182141748838400)]} 0 2
[junit4:junit4] 2> 108793 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10077 (1438182141749886976)]} 0 2
[junit4:junit4] 2> 108797 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[77 (1438182141754081280)]} 0 2
[junit4:junit4] 2> 108799 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10077 (1438182141756178432)]} 0 2
[junit4:junit4] 2> 108852 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[63 (-1438182141812801536)]} 0 0
[junit4:junit4] 2> 108854 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10063 (-1438182141814898688)]} 0 1
[junit4:junit4] 2> 108856 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[63 (-1438182141816995840)]} 0 1
[junit4:junit4] 2> 108858 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10063 (-1438182141819092992)]} 0 1
[junit4:junit4] 2> 108861 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[78 (1438182141821190144)]} 0 2
[junit4:junit4] 2> 108863 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10078 (1438182141823287296)]} 0 2
[junit4:junit4] 2> 108866 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[78 (1438182141826433024)]} 0 2
[junit4:junit4] 2> 108868 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10078 (1438182141828530176)]} 0 2
[junit4:junit4] 2> 108883 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[79 (1438182141844258816)]} 0 1
[junit4:junit4] 2> 108885 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10079 (1438182141846355968)]} 0 2
[junit4:junit4] 2> 108889 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[79 (1438182141850550272)]} 0 2
[junit4:junit4] 2> 108891 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10079 (1438182141852647424)]} 0 2
[junit4:junit4] 2> 108977 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[80 (1438182141941776384)]} 0 2
[junit4:junit4] 2> 108978 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10080 (1438182141943873536)]} 0 2
[junit4:junit4] 2> 108982 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[80 (1438182141948067840)]} 0 2
[junit4:junit4] 2> 108984 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10080 (1438182141950164992)]} 0 2
[junit4:junit4] 2> 108998 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[81 (1438182141964845056)]} 0 2
[junit4:junit4] 2> 109000 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10081 (1438182141966942208)]} 0 2
[junit4:junit4] 2> 109003 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[81 (1438182141970087936)]} 0 2
[junit4:junit4] 2> 109005 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10081 (1438182141972185088)]} 0 2
[junit4:junit4] 2> 109053 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[64 (-1438182142023565312)]} 0 0
[junit4:junit4] 2> 109055 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10064 (-1438182142025662464)]} 0 1
[junit4:junit4] 2> 109058 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[64 (-1438182142028808192)]} 0 1
[junit4:junit4] 2> 109059 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10064 (-1438182142029856768)]} 0 1
[junit4:junit4] 2> 109063 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[82 (1438182142033002496)]} 0 2
[junit4:junit4] 2> 109064 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10082 (1438182142034051072)]} 0 2
[junit4:junit4] 2> 109069 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10082 (1438182142039293952)]} 0 2
[junit4:junit4] 2> 109069 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[82 (1438182142039293953)]} 0 2
[junit4:junit4] 2> 109126 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10083 (1438182142098014208)]} 0 2
[junit4:junit4] 2> 109126 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[83 (1438182142099062784)]} 0 2
[junit4:junit4] 2> 109131 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10083 (1438182142104305664)]} 0 2
[junit4:junit4] 2> 109132 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[83 (1438182142105354240)]} 0 2
[junit4:junit4] 2> 109184 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10084 (1438182142158831616)]} 0 2
[junit4:junit4] 2> 109184 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[84 (1438182142159880192)]} 0 2
[junit4:junit4] 2> 109189 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10084 (1438182142165123072)]} 0 2
[junit4:junit4] 2> 109190 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[84 (1438182142166171648)]} 0 2
[junit4:junit4] 2> 109240 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10065 (-1438182142219649024)]} 0 0
[junit4:junit4] 2> 109241 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[65 (-1438182142219649025)]} 0 1
[junit4:junit4] 2> 109245 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10065 (-1438182142224891904)]} 0 1
[junit4:junit4] 2> 109245 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[65 (-1438182142224891905)]} 0 1
[junit4:junit4] 2> 109250 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10085 (1438182142229086208)]} 0 2
[junit4:junit4] 2> 109251 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[85 (1438182142230134784)]} 0 2
[junit4:junit4] 2> 109255 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10085 (1438182142234329088)]} 0 2
[junit4:junit4] 2> 109257 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[85 (1438182142236426240)]} 0 2
[junit4:junit4] 2> 109346 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10066 (-1438182142330798080)]} 0 1
[junit4:junit4] 2> 109347 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[66 (-1438182142331846656)]} 0 0
[junit4:junit4] 2> 109350 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10066 (-1438182142334992384)]} 0 1
[junit4:junit4] 2> 109350 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[66 (-1438182142334992385)]} 0 0
[junit4:junit4] 2> 109355 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[86 (1438182142338138112)]} 0 1
[junit4:junit4] 2> 109355 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10086 (1438182142338138113)]} 0 2
[junit4:junit4] 2> 109359 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10086 (1438182142343380992)]} 0 1
[junit4:junit4] 2> 109359 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[86 (1438182142343380993)]} 0 1
[junit4:junit4] 2> 109389 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10067 (-1438182142375886848)]} 0 0
[junit4:junit4] 2> 109389 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[67 (-1438182142375886849)]} 0 0
[junit4:junit4] 2> 109392 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10067 (-1438182142379032576)]} 0 0
[junit4:junit4] 2> 109393 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[67 (-1438182142380081152)]} 0 0
[junit4:junit4] 2> 109396 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10087 (1438182142382178304)]} 0 1
[junit4:junit4] 2> 109397 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[87 (1438182142383226880)]} 0 1
[junit4:junit4] 2> 109400 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10087 (1438182142386372608)]} 0 1
[junit4:junit4] 2> 109401 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[87 (1438182142387421184)]} 0 1
[junit4:junit4] 2> 109503 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10088 (1438182142494375936)]} 0 2
[junit4:junit4] 2> 109504 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[88 (1438182142495424512)]} 0 2
[junit4:junit4] 2> 109507 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10088 (1438182142498570240)]} 0 1
[junit4:junit4] 2> 109509 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[88 (1438182142500667392)]} 0 2
[junit4:junit4] 2> 109562 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10071 (-1438182142557290496)]} 0 1
[junit4:junit4] 2> 109563 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[71 (-1438182142558339072)]} 0 0
[junit4:junit4] 2> 109566 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10071 (-1438182142560436224)]} 0 1
[junit4:junit4] 2> 109566 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[71 (-1438182142561484800)]} 0 0
[junit4:junit4] 2> 109570 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10089 (1438182142565679104)]} 0 1
[junit4:junit4] 2> 109570 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[89 (1438182142565679105)]} 0 1
[junit4:junit4] 2> 109575 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10089 (1438182142569873408)]} 0 1
[junit4:junit4] 2> 109576 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[89 (1438182142570921984)]} 0 2
[junit4:junit4] 2> 109593 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10072 (-1438182142588747776)]} 0 1
[junit4:junit4] 2> 109594 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[72 (-1438182142589796352)]} 0 1
[junit4:junit4] 2> 109597 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[72 (-1438182142593990656)]} 0 1
[junit4:junit4] 2> 109597 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10072 (-1438182142593990657)]} 0 1
[junit4:junit4] 2> 109601 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10090 (1438182142597136384)]} 0 1
[junit4:junit4] 2> 109602 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[90 (1438182142598184960)]} 0 2
[junit4:junit4] 2> 109606 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10090 (1438182142602379264)]} 0 2
[junit4:junit4] 2> 109606 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[90 (1438182142602379265)]} 0 1
[junit4:junit4] 2> 109710 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10091 (1438182142712479744)]} 0 1
[junit4:junit4] 2> 109711 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[91 (1438182142712479745)]} 0 2
[junit4:junit4] 2> 109715 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10091 (1438182142716674048)]} 0 2
[junit4:junit4] 2> 109716 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[91 (1438182142716674049)]} 0 2
[junit4:junit4] 2> 109734 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10076 (-1438182142736596992)]} 0 1
[junit4:junit4] 2> 109735 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[76 (-1438182142737645568)]} 0 1
[junit4:junit4] 2> 109737 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10076 (-1438182142740791296)]} 0 0
[junit4:junit4] 2> 109738 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[76 (-1438182142741839872)]} 0 0
[junit4:junit4] 2> 109741 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10092 (1438182142743937024)]} 0 1
[junit4:junit4] 2> 109743 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[92 (1438182142746034176)]} 0 2
[junit4:junit4] 2> 109745 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10092 (1438182142748131328)]} 0 1
[junit4:junit4] 2> 109747 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[92 (1438182142750228480)]} 0 1
[junit4:junit4] 2> 109831 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10093 (1438182142838308864)]} 0 2
[junit4:junit4] 2> 109832 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[93 (1438182142839357440)]} 0 1
[junit4:junit4] 2> 109836 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10093 (1438182142843551744)]} 0 1
[junit4:junit4] 2> 109836 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[93 (1438182142843551745)]} 0 1
[junit4:junit4] 2> 109872 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[81 (-1438182142882349056)]} 0 0
[junit4:junit4] 2> 109872 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10081 (-1438182142882349057)]} 0 0
[junit4:junit4] 2> 109876 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[81 (-1438182142885494784)]} 0 1
[junit4:junit4] 2> 109876 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10081 (-1438182142886543360)]} 0 0
[junit4:junit4] 2> 109880 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[94 (1438182142889689088)]} 0 1
[junit4:junit4] 2> 109880 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10094 (1438182142889689089)]} 0 1
[junit4:junit4] 2> 109885 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[94 (1438182142894931968)]} 0 2
[junit4:junit4] 2> 109885 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10094 (1438182142894931969)]} 0 1
[junit4:junit4] 2> 109987 T137 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10084 (-1438182143001886721)]} 0 1
[junit4:junit4] 2> 109987 T136 C19 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[84 (-1438182143001886720)]} 0 1
[junit4:junit4] 2> 109990 T158 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10084 (-1438182143006081024)]} 0 0
[junit4:junit4] 2> 109991 T157 C20 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[84 (-1438182143007129600)]} 0 1
[junit4:junit4] 2> ASYNC NEW_CORE C21 name=collection1 org.apache.solr.core.SolrCore@64af4b04 url=http://127.0.0.1:63938/yu/collection1 node=127.0.0.1:63938_yu C21_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:63938_yu, base_url=http://127.0.0.1:63938/yu, leader=true}
[junit4:junit4] 2> 110004 T136 C21 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10095 (1438182143009226752)]} 0 11
[junit4:junit4] 2> 110005 T137 C21 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[95 (1438182143020761088)]} 0 1
[junit4:junit4] 2> ASYNC NEW_CORE C22 name=collection1 org.apache.solr.core.SolrCore@186bd87 url=http://127.0.0.1:63942/yu/collection1 node=127.0.0.1:63942_yu C22_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:63942_yu, base_url=http://127.0.0.1:63942/yu, leader=true}
[junit4:junit4] 2> 110008 T157 C22 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10095 (1438182143024955392)]} 0 1
[junit4:junit4] 2> 110010 T158 C22 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[95 (1438182143026003968)]} 0 1
[junit4:junit4] 2> 110056 T137 C21 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10096 (1438182143074238464)]} 0 1
[junit4:junit4] 2> 110058 T136 C21 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[96 (1438182143076335616)]} 0 1
[junit4:junit4] 2> 110060 T158 C22 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10096 (1438182143078432768)]} 0 1
[junit4:junit4] 2> 110062 T157 C22 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[96 (1438182143080529920)]} 0 1
[junit4:junit4] 2> 110158 T136 C21 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10085 (-1438182143181193216)]} 0 0
[junit4:junit4] 2> 110160 T137 C21 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[85 (-1438182143184338944)]} 0 1
[junit4:junit4] 2> 110160 T157 C22 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[10085 (-1438182143184338944)]} 0 0
[junit4:junit4] 2> 110163 T158 C22 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {delete=[85 (-1438182143187484672)]} 0 0
[junit4:junit4] 2> 110164 T137 C21 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10097 (1438182143187484672)]} 0 1
[junit4:junit4] 2> 110167 T136 C21 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[97 (1438182143190630400)]} 0 1
[junit4:junit4] 2> 110168 T158 C22 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10097 (1438182143191678976)]} 0 1
[junit4:junit4] 2> 110171 T157 C22 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[97 (1438182143194824704)]} 0 1
[junit4:junit4] 2> 110217 T136 C21 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10098 (1438182143243059200)]} 0 1
[junit4:junit4] 2> 110221 T137 C21 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[98 (1438182143247253504)]} 0 2
[junit4:junit4] 2> 110221 T157 C22 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10098 (1438182143247253504)]} 0 1
[junit4:junit4] 2> 110226 T158 C22 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[98 (1438182143252496384)]} 0 2
[junit4:junit4] 2> 110323 T137 C21 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[10099 (1438182143354208256)]} 0 2
[junit4:junit4] 2> 110328 T136 C21 P63938 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[99 (1438182143359451136)]} 0 2
[junit4:junit4] 2> 110329 T158 C22 P63942 oasup.LogUpdateProcessor.finish [collection1] webapp=/yu path=/update params={wt=javabin&version=2} {add=[100
[...truncated too long message...]
ocsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=785,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4:junit4] 2> 1980270 T3885 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 1980270 T3885 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 1980271 T3885 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 1980274 T3885 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 1980274 T3885 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 1980274 T3885 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371559013460/jetty4 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371559013460/jetty4;done=false>>]
[junit4:junit4] 2> 1980275 T3885 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371559013460/jetty4
[junit4:junit4] 2> 1980275 T3885 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371559013460/jetty4/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371559013460/jetty4/index;done=false>>]
[junit4:junit4] 2> 1980275 T3885 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371559013460/jetty4/index
[junit4:junit4] 2> 1980275 T3885 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 1980276 T3885 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard2/election/89886491506507788-127.0.0.1:15635__collection1-n_0000000001
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4] 2> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:152)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:149)
[junit4:junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:149)
[junit4:junit4] 2> at org.apache.solr.cloud.ElectionContext.cancelElection(ElectionContext.java:63)
[junit4:junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:260)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:156)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:100)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4] 2>
[junit4:junit4] 2> 1980277 T3885 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 1980277 T3885 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1980277 T3885 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 1980278 T3885 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1980278 T3885 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 1980278 T3885 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:235)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:232)
[junit4:junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:232)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4] 2>
[junit4:junit4] 2> NOTE: test params are: codec=Lucene41, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=mt, timezone=MET
[junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=3,free=233000944,total=429916160
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TermVectorComponentTest, TestCoreContainer, TestSort, ShowFileRequestHandlerTest, TestPerFieldSimilarity, TestPseudoReturnFields, ReturnFieldsTest, ExternalFileFieldSortTest, TestValueSourceCache, TestReloadAndDeleteDocs, TestSchemaResource, AutoCommitTest, DirectUpdateHandlerOptimizeTest, RecoveryZkTest, ResponseLogComponentTest, TestIBSimilarityFactory, TestReversedWildcardFilterFactory, TestStressVersions, ZkControllerTest, TestXIncludeConfig, SpellingQueryConverterTest, TestLRUCache, WordBreakSolrSpellCheckerTest, ChaosMonkeyNothingIsSafeTest, ShardSplitTest, TestSearchPerf, SuggesterTest, UnloadDistributedZkTest, MoreLikeThisHandlerTest, PolyFieldTest, TestFaceting, TestPropInjectDefaults, MBeansHandlerTest, QueryEqualityTest, TestClassNameShortening, TestSolrCoreProperties, TestFastLRUCache, TestPHPSerializedResponseWriter, MinimalSchemaTest, UpdateRequestProcessorFactoryTest, TestSolrDeletionPolicy1, StatelessScriptUpdateProcessorFactoryTest, DistributedQueryElevationComponentTest, SolrInfoMBeanTest, ScriptEngineTest, CacheHeaderTest, PrimUtilsTest, SuggesterFSTTest, SchemaVersionSpecificBehaviorTest, SolrCoreTest, TestRealTimeGet, TestFieldCollectionResource, TestReplicationHandler, DocumentBuilderTest, SpatialFilterTest, PreAnalyzedFieldTest, LukeRequestHandlerTest, TestConfig, TestUniqueKeyFieldResource, DistributedTermsComponentTest, PeerSyncTest, IndexSchemaTest, SOLR749Test, ZkCLITest, TestCloudManagedSchema, QueryParsingTest, CircularListTest, JsonLoaderTest, TestQuerySenderListener, TestSurroundQueryParser, TestDocumentBuilder, SimpleFacetsTest, BasicDistributedZk2Test, OverseerTest, LeaderElectionIntegrationTest, ClusterStateUpdateTest, TestRandomFaceting, ShardRoutingCustomTest, SliceStateUpdateTest, TestDistributedSearch, TestDistributedGrouping, TestRecovery, DistributedSpellCheckComponentTest, TestJoin, ConvertedLegacyTest, TestFiltering, SignatureUpdateProcessorFactoryTest, CoreAdminHandlerTest, SuggesterTSTTest, TestTrie, SuggesterWFSTTest, NoCacheHeaderTest, TestUpdate, DefaultValueUpdateProcessorTest, DocValuesMultiTest, DisMaxRequestHandlerTest, TestWriterPerf, TestOmitPositions, TermsComponentTest, TestSolrQueryParser, IndexSchemaRuntimeFieldTest, RegexBoostProcessorTest, UniqFieldsUpdateProcessorFactoryTest, TestPartialUpdateDeduplication, CSVRequestHandlerTest, TestComponentsName, TestBinaryResponseWriter, HighlighterConfigTest, AlternateDirectoryTest, TestQuerySenderNoQuery, TestStressRecovery, ChaosMonkeyShardSplitTest]
[junit4:junit4] Completed on J0 in 229.30s, 1 test, 1 error <<< FAILURES!
[...truncated 13 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:386: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:366: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:181: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:437: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1243: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:887: There were test failures: 297 suites, 1250 tests, 1 suite-level error, 2 errors, 1 failure, 13 ignored (7 assumptions)
Total time: 53 minutes 32 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure