You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/02/09 11:07:32 UTC

[JENKINS] Lucene-Solr-4.x-Linux (32bit/ibm-j9-jdk7) - Build # 9310 - Failure!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/9310/
Java: 32bit/ibm-j9-jdk7 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}

4 tests failed.
REGRESSION:  org.apache.solr.TestDistributedSearch.testDistribSearch

Error Message:
Expected the partialResults header to be null

Stack Trace:
java.lang.AssertionError: Expected the partialResults header to be null
	at __randomizedtesting.SeedInfo.seed([2696C0E878F27C17:A7704EF00FAD1C2B]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.TestDistributedSearch.validateControlData(TestDistributedSearch.java:525)
	at org.apache.solr.TestDistributedSearch.queryPartialResults(TestDistributedSearch.java:445)
	at org.apache.solr.TestDistributedSearch.doTest(TestDistributedSearch.java:379)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:873)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:88)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
	at java.lang.reflect.Method.invoke(Method.java:613)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:780)


REGRESSION:  org.apache.solr.handler.component.DistributedQueryElevationComponentTest.testDistribSearch

Error Message:
.responseHeader.params.version!=rows (unordered or missing)

Stack Trace:
junit.framework.AssertionFailedError: .responseHeader.params.version!=rows (unordered or missing)
	at __randomizedtesting.SeedInfo.seed([2696C0E878F27C17:A7704EF00FAD1C2B]:0)
	at junit.framework.Assert.fail(Assert.java:50)
	at org.apache.solr.BaseDistributedSearchTestCase.compareSolrResponses(BaseDistributedSearchTestCase.java:840)
	at org.apache.solr.BaseDistributedSearchTestCase.compareResponses(BaseDistributedSearchTestCase.java:859)
	at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:562)
	at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:542)
	at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:521)
	at org.apache.solr.handler.component.DistributedQueryElevationComponentTest.doTest(DistributedQueryElevationComponentTest.java:81)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:867)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:88)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
	at java.lang.reflect.Method.invoke(Method.java:613)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:780)


REGRESSION:  org.apache.solr.handler.component.TermVectorComponentDistributedTest.testDistribSearch

Error Message:
.termVectors.0.test_offtv!=test_basictv (unordered or missing)

Stack Trace:
junit.framework.AssertionFailedError: .termVectors.0.test_offtv!=test_basictv (unordered or missing)
	at __randomizedtesting.SeedInfo.seed([2696C0E878F27C17:A7704EF00FAD1C2B]:0)
	at junit.framework.Assert.fail(Assert.java:50)
	at org.apache.solr.BaseDistributedSearchTestCase.compareSolrResponses(BaseDistributedSearchTestCase.java:840)
	at org.apache.solr.BaseDistributedSearchTestCase.compareResponses(BaseDistributedSearchTestCase.java:859)
	at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:562)
	at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:542)
	at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:521)
	at org.apache.solr.handler.component.TermVectorComponentDistributedTest.doTest(TermVectorComponentDistributedTest.java:127)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:873)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:88)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
	at java.lang.reflect.Method.invoke(Method.java:613)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:780)


REGRESSION:  org.apache.solr.update.processor.SignatureUpdateProcessorFactoryTest.testDupeDetection

Error Message:
expected:<1> but was:<2>

Stack Trace:
java.lang.AssertionError: expected:<1> but was:<2>
	at __randomizedtesting.SeedInfo.seed([2696C0E878F27C17:7DA63B207BB5DF6B]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.junit.Assert.assertEquals(Assert.java:456)
	at org.apache.solr.update.processor.SignatureUpdateProcessorFactoryTest.checkNumDocs(SignatureUpdateProcessorFactoryTest.java:71)
	at org.apache.solr.update.processor.SignatureUpdateProcessorFactoryTest.testDupeDetection(SignatureUpdateProcessorFactoryTest.java:119)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:88)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
	at java.lang.reflect.Method.invoke(Method.java:613)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:780)




Build Log:
[...truncated 10254 lines...]
   [junit4] Suite: org.apache.solr.update.processor.SignatureUpdateProcessorFactoryTest
   [junit4]   2> 215597 T892 oas.SolrTestCaseJ4.initCore ####initCore
   [junit4]   2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-SignatureUpdateProcessorFactoryTest-1391939118661
   [junit4]   2> 215597 T892 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/'
   [junit4]   2> 215598 T892 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 215599 T892 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 215643 T892 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 215701 T892 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 215702 T892 oass.IndexSchema.readSchema Reading Solr Schema from schema12.xml
   [junit4]   2> 215709 T892 oass.IndexSchema.readSchema [null] Schema name=test
   [junit4]   2> 216127 T892 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 216131 T892 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 216133 T892 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 216142 T892 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 216146 T892 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 216149 T892 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 216150 T892 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr
   [junit4]   2> 216150 T892 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/'
   [junit4]   2> 216159 T892 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 216160 T892 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr
   [junit4]   2> 216160 T892 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/'
   [junit4]   2> 216208 T892 oasc.CoreContainer.<init> New CoreContainer -1899279957
   [junit4]   2> 216209 T892 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/]
   [junit4]   2> 216211 T892 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
   [junit4]   2> 216211 T892 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 216212 T892 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
   [junit4]   2> 216213 T892 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 216213 T892 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 216214 T892 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 216215 T892 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 216215 T892 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 216216 T892 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 216223 T892 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 216224 T892 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 216224 T892 oasc.CoreContainer.load Host Name: 
   [junit4]   2> 216229 T893 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1
   [junit4]   2> 216229 T893 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/'
   [junit4]   2> 216230 T893 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 216230 T893 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 216275 T893 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 216331 T893 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 216332 T893 oass.IndexSchema.readSchema Reading Solr Schema from schema12.xml
   [junit4]   2> 216338 T893 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 216725 T893 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 216729 T893 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 216732 T893 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 216747 T893 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 216753 T893 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 216759 T893 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 216760 T893 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-SignatureUpdateProcessorFactoryTest-1391939118661/
   [junit4]   2> 216760 T893 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@381ddd72
   [junit4]   2> 216762 T893 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-SignatureUpdateProcessorFactoryTest-1391939118661
   [junit4]   2> 216762 T893 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-SignatureUpdateProcessorFactoryTest-1391939118661/index/
   [junit4]   2> 216763 T893 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-SignatureUpdateProcessorFactoryTest-1391939118661/index' doesn't exist. Creating new index...
   [junit4]   2> 216764 T893 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-SignatureUpdateProcessorFactoryTest-1391939118661/index
   [junit4]   2> 216764 T893 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=41, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 216766 T893 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_1,generation=1}
   [junit4]   2> 216766 T893 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 216767 T893 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
   [junit4]   2> 216770 T893 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 216771 T893 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 216772 T893 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 216772 T893 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 216772 T893 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 216773 T893 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 216773 T893 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 216773 T893 oasup.UniqFieldsUpdateProcessorFactory.init WARN Use of the 'fields' init param in UniqFieldsUpdateProcessorFactory is deprecated, please use 'fieldName' (or another FieldMutatingUpdateProcessorFactory selector option) instead
   [junit4]   2> 216774 T893 oasup.UniqFieldsUpdateProcessorFactory.init Replacing 'fields' init param with (individual) 'fieldName' params
   [junit4]   2> 216774 T893 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 216774 T893 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 216775 T893 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 216775 T893 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 216776 T893 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 216776 T893 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 216776 T893 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 216777 T893 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 216777 T893 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 216778 T893 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
   [junit4]   2> 216778 T893 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
   [junit4]   2> 216779 T893 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 216779 T893 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 216779 T893 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
   [junit4]   2> 216780 T893 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
   [junit4]   2> 216780 T893 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
   [junit4]   2> 216781 T893 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 216781 T893 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 216781 T893 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 216782 T893 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 216782 T893 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 216783 T893 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 216783 T893 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 216784 T893 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 216784 T893 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 216785 T893 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
   [junit4]   2> 216785 T893 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
   [junit4]   2> 216787 T893 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 216788 T893 oasc.SolrCore.initDeprecatedSupport WARN solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
   [junit4]   2> 216790 T893 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 216790 T893 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 216791 T893 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=31, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 216792 T893 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_1,generation=1}
   [junit4]   2> 216792 T893 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 216792 T893 oass.SolrIndexSearcher.<init> Opening Searcher@ea89d514[collection1] main
   [junit4]   2> 216793 T893 oashc.SpellCheckComponent.inform Initializing spell checkers
   [junit4]   2> 216857 T893 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 216883 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@ea89d514[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 216886 T893 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 216888 T892 oas.SolrTestCaseJ4.initCore ####initCore end
   [junit4]   2> 216893 T892 oas.SolrTestCaseJ4.setUp ###Starting testNonIndexedSignatureField
   [junit4]   2> ASYNC  NEW_CORE C287 name=collection1 org.apache.solr.core.SolrCore@c1f8cbbb
   [junit4]   2> 216893 T892 C287 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 216894 T892 C287 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {deleteByQuery=*:*} 0 1
   [junit4]   2> 216898 T892 C287 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 216899 T892 C287 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_1,generation=1}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_2,generation=2}
   [junit4]   2> 216900 T892 C287 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
   [junit4]   2> 216901 T892 C287 oass.SolrIndexSearcher.<init> Opening Searcher@8a87bd30[collection1] main
   [junit4]   2> 216901 T892 C287 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 216903 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@8a87bd30[collection1] main{StandardDirectoryReader(segments_2:2:nrt)}
   [junit4]   2> 216903 T892 C287 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {commit=} 0 5
   [junit4]   2> 216912 T892 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 216915 T892 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_2,generation=2}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_3,generation=3}
   [junit4]   2> 216915 T892 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
   [junit4]   2> 216917 T892 oass.SolrIndexSearcher.<init> Opening Searcher@3254ae45[collection1] main
   [junit4]   2> 216918 T892 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 216919 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3254ae45[collection1] main{StandardDirectoryReader(segments_3:4:nrt _0(4.7):c2)}
   [junit4]   2> 216920 T892 oas.SolrTestCaseJ4.tearDown ###Ending testNonIndexedSignatureField
   [junit4]   2> 216925 T892 oas.SolrTestCaseJ4.setUp ###Starting testDupeDetection
   [junit4]   2> 217114 T892 C287 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 217115 T892 C287 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {deleteByQuery=*:*} 0 190
   [junit4]   2> 217300 T892 C287 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217303 T892 C287 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_3,generation=3}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_4,generation=4}
   [junit4]   2> 217305 T892 C287 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
   [junit4]   2> 217308 T892 C287 oass.SolrIndexSearcher.<init> Opening Searcher@33fc1c3b[collection1] main
   [junit4]   2> 217309 T892 C287 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217310 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@33fc1c3b[collection1] main{StandardDirectoryReader(segments_4:5:nrt)}
   [junit4]   2> 217311 T892 C287 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {commit=} 0 11
   [junit4]   2> 217318 T892 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217322 T892 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_4,generation=4}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_5,generation=5}
   [junit4]   2> 217323 T892 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
   [junit4]   2> 217324 T892 oass.SolrIndexSearcher.<init> Opening Searcher@978d93d3[collection1] main
   [junit4]   2> 217325 T892 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217327 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@978d93d3[collection1] main{StandardDirectoryReader(segments_5:7:nrt _1(4.7):c2)}
   [junit4]   2> 217333 T892 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217338 T892 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_5,generation=5}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_6,generation=6}
   [junit4]   2> 217338 T892 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
   [junit4]   2> 217339 T892 oass.SolrIndexSearcher.<init> Opening Searcher@ae6c134f[collection1] main
   [junit4]   2> 217340 T892 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217342 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@ae6c134f[collection1] main{StandardDirectoryReader(segments_6:10:nrt _1(4.7):c2/1:delGen=1 _2(4.7):c1)}
   [junit4]   2> 217343 T892 oas.SolrTestCaseJ4.tearDown ###Ending testDupeDetection
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=SignatureUpdateProcessorFactoryTest -Dtests.method=testDupeDetection -Dtests.seed=2696C0E878F27C17 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=tr_TR -Dtests.timezone=Asia/Singapore -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 0.44s J1 | SignatureUpdateProcessorFactoryTest.testDupeDetection <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: expected:<1> but was:<2>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([2696C0E878F27C17:7DA63B207BB5DF6B]:0)
   [junit4]    > 	at org.apache.solr.update.processor.SignatureUpdateProcessorFactoryTest.checkNumDocs(SignatureUpdateProcessorFactoryTest.java:71)
   [junit4]    > 	at org.apache.solr.update.processor.SignatureUpdateProcessorFactoryTest.testDupeDetection(SignatureUpdateProcessorFactoryTest.java:119)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:780)
   [junit4]   2> 217367 T892 oas.SolrTestCaseJ4.setUp ###Starting testMultiThreaded
   [junit4]   2> 217368 T892 C287 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 217369 T892 C287 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {deleteByQuery=*:*} 0 1
   [junit4]   2> 217371 T892 C287 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217372 T892 C287 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_6,generation=6}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_7,generation=7}
   [junit4]   2> 217373 T892 C287 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 7
   [junit4]   2> 217373 T892 C287 oass.SolrIndexSearcher.<init> Opening Searcher@5495d3c[collection1] main
   [junit4]   2> 217374 T892 C287 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217377 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5495d3c[collection1] main{StandardDirectoryReader(segments_7:11:nrt)}
   [junit4]   2> 217378 T892 C287 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {commit=} 0 7
   [junit4]   2> 217391 T904 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217398 T904 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_7,generation=7}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_8,generation=8}
   [junit4]   2> 217399 T904 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 8
   [junit4]   2> 217412 T904 oass.SolrIndexSearcher.<init> Opening Searcher@e6e0a13[collection1] main
   [junit4]   2> 217413 T904 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217414 T903 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217415 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@e6e0a13[collection1] main{StandardDirectoryReader(segments_8:18:nrt _4(4.7):c12/11:delGen=1)}
   [junit4]   2> 217431 T903 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_8,generation=8}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_9,generation=9}
   [junit4]   2> 217432 T903 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 9
   [junit4]   2> 217444 T903 oass.SolrIndexSearcher.<init> Opening Searcher@407c48c3[collection1] main
   [junit4]   2> 217445 T903 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217446 T905 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217450 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@407c48c3[collection1] main{StandardDirectoryReader(segments_9:26:nrt _6(4.7):c37/36:delGen=1)}
   [junit4]   2> 217463 T905 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_9,generation=9}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_a,generation=10}
   [junit4]   2> 217464 T905 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 10
   [junit4]   2> 217477 T905 oass.SolrIndexSearcher.<init> Opening Searcher@ab205184[collection1] main
   [junit4]   2> 217479 T905 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217479 T904 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217480 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@ab205184[collection1] main{StandardDirectoryReader(segments_a:34:nrt _8(4.7):c22/21:delGen=1)}
   [junit4]   2> 217493 T904 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_a,generation=10}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_b,generation=11}
   [junit4]   2> 217493 T904 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 11
   [junit4]   2> 217506 T904 oass.SolrIndexSearcher.<init> Opening Searcher@d97b6143[collection1] main
   [junit4]   2> 217507 T904 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217507 T903 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217508 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@d97b6143[collection1] main{StandardDirectoryReader(segments_b:42:nrt _a(4.7):c24/22:delGen=1)}
   [junit4]   2> 217520 T903 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_b,generation=11}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_c,generation=12}
   [junit4]   2> 217520 T903 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 12
   [junit4]   2> 217529 T903 oass.SolrIndexSearcher.<init> Opening Searcher@957cc65e[collection1] main
   [junit4]   2> 217530 T903 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217530 T905 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217531 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@957cc65e[collection1] main{StandardDirectoryReader(segments_c:50:nrt _c(4.7):c14/12:delGen=1)}
   [junit4]   2> 217535 T905 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_c,generation=12}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_d,generation=13}
   [junit4]   2> 217535 T905 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 13
   [junit4]   2> 217540 T905 oass.SolrIndexSearcher.<init> Opening Searcher@f3f1e638[collection1] main
   [junit4]   2> 217541 T905 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217541 T904 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217542 T904 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_d,generation=13}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_e,generation=14}
   [junit4]   2> 217542 T904 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 14
   [junit4]   2> 217542 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@f3f1e638[collection1] main{StandardDirectoryReader(segments_d:56:nrt _c(4.7):c14/13:delGen=2 _e(4.7):c1)}
   [junit4]   2> 217543 T904 oass.SolrIndexSearcher.<init> Opening Searcher@aeb74afc[collection1] main
   [junit4]   2> 217544 T904 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217544 T903 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217545 T903 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 217545 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@aeb74afc[collection1] main{StandardDirectoryReader(segments_d:56:nrt _c(4.7):c14/13:delGen=2 _e(4.7):c1)}
   [junit4]   2> 217549 T903 oass.SolrIndexSearcher.<init> Opening Searcher@7e29f70b[collection1] main
   [junit4]   2> 217549 T903 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217550 T904 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217550 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7e29f70b[collection1] main{StandardDirectoryReader(segments_e:61:nrt _e(4.7):c1 _f(4.7):c1)}
   [junit4]   2> 217553 T904 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_e,generation=14}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_f,generation=15}
   [junit4]   2> 217553 T904 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 15
   [junit4]   2> 217556 T904 oass.SolrIndexSearcher.<init> Opening Searcher@821373ea[collection1] main
   [junit4]   2> 217557 T904 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217557 T905 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217558 T905 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_f,generation=15}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_g,generation=16}
   [junit4]   2> 217558 T905 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 16
   [junit4]   2> 217559 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@821373ea[collection1] main{StandardDirectoryReader(segments_f:68:nrt _h(4.7):c1)}
   [junit4]   2> 217560 T905 oass.SolrIndexSearcher.<init> Opening Searcher@c2e37472[collection1] main
   [junit4]   2> 217560 T905 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217561 T903 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217561 T903 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 217562 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@c2e37472[collection1] main{StandardDirectoryReader(segments_f:68:nrt _h(4.7):c1)}
   [junit4]   2> 217566 T903 oass.SolrIndexSearcher.<init> Opening Searcher@bb2f89b2[collection1] main
   [junit4]   2> 217567 T903 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217568 T904 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217569 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@bb2f89b2[collection1] main{StandardDirectoryReader(segments_g:71:nrt _h(4.7):c1 _i(4.7):c1)}
   [junit4]   2> 217571 T904 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_g,generation=16}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_h,generation=17}
   [junit4]   2> 217571 T904 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 17
   [junit4]   2> 217572 T904 oass.SolrIndexSearcher.<init> Opening Searcher@e85fb426[collection1] main
   [junit4]   2> 217573 T904 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217573 T905 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217574 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@e85fb426[collection1] main{StandardDirectoryReader(segments_h:74:nrt _i(4.7):c1 _j(4.7):c1)}
   [junit4]   2> 217577 T905 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_h,generation=17}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_i,generation=18}
   [junit4]   2> 217578 T905 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 18
   [junit4]   2> 217581 T905 oass.SolrIndexSearcher.<init> Opening Searcher@387829a6[collection1] main
   [junit4]   2> 217582 T905 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217582 T903 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217583 T903 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_i,generation=18}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_j,generation=19}
   [junit4]   2> 217583 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@387829a6[collection1] main{StandardDirectoryReader(segments_i:80:nrt _k(4.7):c1 _l(4.7):c1)}
   [junit4]   2> 217584 T903 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 19
   [junit4]   2> 217585 T903 oass.SolrIndexSearcher.<init> Opening Searcher@1fa2cd0b[collection1] main
   [junit4]   2> 217585 T903 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217586 T904 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217587 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1fa2cd0b[collection1] main{StandardDirectoryReader(segments_i:80:nrt _k(4.7):c1 _l(4.7):c1)}
   [junit4]   2> 217591 T904 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_j,generation=19}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_k,generation=20}
   [junit4]   2> 217591 T904 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 20
   [junit4]   2> 217594 T904 oass.SolrIndexSearcher.<init> Opening Searcher@76a80551[collection1] main
   [junit4]   2> 217595 T904 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217595 T905 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217596 T905 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_k,generation=20}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_l,generation=21}
   [junit4]   2> 217596 T905 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 21
   [junit4]   2> 217596 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@76a80551[collection1] main{StandardDirectoryReader(segments_k:88:nrt _n(4.7):c1)}
   [junit4]   2> 217597 T905 oass.SolrIndexSearcher.<init> Opening Searcher@dd592738[collection1] main
   [junit4]   2> 217598 T905 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217598 T903 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217598 T903 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 217599 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@dd592738[collection1] main{StandardDirectoryReader(segments_k:88:nrt _n(4.7):c1)}
   [junit4]   2> 217603 T903 oass.SolrIndexSearcher.<init> Opening Searcher@451ea313[collection1] main
   [junit4]   2> 217603 T903 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217604 T904 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217605 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@451ea313[collection1] main{StandardDirectoryReader(segments_l:93:nrt _o(4.7):c1)}
   [junit4]   2> 217606 T904 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_l,generation=21}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_m,generation=22}
   [junit4]   2> 217607 T904 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 22
   [junit4]   2> 217607 T904 oass.SolrIndexSearcher.<init> Opening Searcher@1edc7b82[collection1] main
   [junit4]   2> 217608 T904 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217608 T905 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217609 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1edc7b82[collection1] main{StandardDirectoryReader(segments_m:96:nrt _p(4.7):c1)}
   [junit4]   2> 217613 T905 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_m,generation=22}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_n,generation=23}
   [junit4]   2> 217613 T905 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 23
   [junit4]   2> 217617 T905 oass.SolrIndexSearcher.<init> Opening Searcher@45457e47[collection1] main
   [junit4]   2> 217618 T905 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217618 T903 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217619 T903 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_n,generation=23}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_o,generation=24}
   [junit4]   2> 217620 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@45457e47[collection1] main{StandardDirectoryReader(segments_n:104:nrt _r(4.7):c1)}
   [junit4]   2> 217620 T903 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 24
   [junit4]   2> 217621 T903 oass.SolrIndexSearcher.<init> Opening Searcher@48a8624c[collection1] main
   [junit4]   2> 217621 T903 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217622 T904 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217622 T904 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 217623 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@48a8624c[collection1] main{StandardDirectoryReader(segments_n:104:nrt _r(4.7):c1)}
   [junit4]   2> 217626 T904 oass.SolrIndexSearcher.<init> Opening Searcher@e76fb2b4[collection1] main
   [junit4]   2> 217627 T904 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217628 T905 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217628 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@e76fb2b4[collection1] main{StandardDirectoryReader(segments_o:109:nrt _s(4.7):c1)}
   [junit4]   2> 217631 T905 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_o,generation=24}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_p,generation=25}
   [junit4]   2> 217631 T905 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 25
   [junit4]   2> 217635 T905 oass.SolrIndexSearcher.<init> Opening Searcher@f9741b61[collection1] main
   [junit4]   2> 217635 T905 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217636 T903 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217637 T903 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_p,generation=25}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_q,generation=26}
   [junit4]   2> 217637 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@f9741b61[collection1] main{StandardDirectoryReader(segments_p:114:nrt _s(4.7):c1 _u(4.7):c1)}
   [junit4]   2> 217637 T903 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 26
   [junit4]   2> 217638 T903 oass.SolrIndexSearcher.<init> Opening Searcher@e62d21a[collection1] main
   [junit4]   2> 217639 T903 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217639 T904 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217640 T904 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 217640 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@e62d21a[collection1] main{StandardDirectoryReader(segments_p:114:nrt _s(4.7):c1 _u(4.7):c1)}
   [junit4]   2> 217694 T904 oass.SolrIndexSearcher.<init> Opening Searcher@92e4a7cd[collection1] main
   [junit4]   2> 217694 T904 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217695 T905 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217696 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@92e4a7cd[collection1] main{StandardDirectoryReader(segments_q:119:nrt _u(4.7):c1 _v(4.7):c1)}
   [junit4]   2> 217699 T905 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_q,generation=26}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_r,generation=27}
   [junit4]   2> 217699 T905 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 27
   [junit4]   2> 217702 T905 oass.SolrIndexSearcher.<init> Opening Searcher@f0fd1c1e[collection1] main
   [junit4]   2> 217703 T905 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217703 T903 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217704 T903 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_r,generation=27}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_s,generation=28}
   [junit4]   2> 217705 T903 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 28
   [junit4]   2> 217704 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@f0fd1c1e[collection1] main{StandardDirectoryReader(segments_r:126:nrt _x(4.7):c1)}
   [junit4]   2> 217706 T903 oass.SolrIndexSearcher.<init> Opening Searcher@f000d6ad[collection1] main
   [junit4]   2> 217707 T903 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217707 T904 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217708 T904 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 217709 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@f000d6ad[collection1] main{StandardDirectoryReader(segments_r:126:nrt _x(4.7):c1)}
   [junit4]   2> 217712 T904 oass.SolrIndexSearcher.<init> Opening Searcher@6917a1cf[collection1] main
   [junit4]   2> 217713 T904 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217714 T905 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217715 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6917a1cf[collection1] main{StandardDirectoryReader(segments_s:129:nrt _x(4.7):c1 _y(4.7):c1)}
   [junit4]   2> 217717 T905 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_s,generation=28}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_t,generation=29}
   [junit4]   2> 217718 T905 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 29
   [junit4]   2> 217718 T905 oass.SolrIndexSearcher.<init> Opening Searcher@4badc108[collection1] main
   [junit4]   2> 217719 T905 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217719 T903 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217720 T903 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 217720 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4badc108[collection1] main{StandardDirectoryReader(segments_t:132:nrt _z(4.7):c1)}
   [junit4]   2> 217721 T903 oass.SolrIndexSearcher.<init> Opening Searcher@9ca0899b[collection1] main
   [junit4]   2> 217721 T903 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217744 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@9ca0899b[collection1] main{StandardDirectoryReader(segments_t:132:nrt _z(4.7):c1)}
   [junit4]   2> 217747 T905 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217750 T905 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_t,generation=29}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_u,generation=30}
   [junit4]   2> 217750 T905 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 30
   [junit4]   2> 217752 T905 oass.SolrIndexSearcher.<init> Opening Searcher@af003c9c[collection1] main
   [junit4]   2> 217753 T905 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217754 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@af003c9c[collection1] main{StandardDirectoryReader(segments_u:136:nrt _10(4.7):c1)}
   [junit4]   2> ASYNC  NEW_CORE C288 name=collection1 org.apache.solr.core.SolrCore@c1f8cbbb
   [junit4]   2> 217755 T892 C288 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217756 T892 C288 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 217757 T892 C288 oass.SolrIndexSearcher.<init> Opening Searcher@72eef1c[collection1] main
   [junit4]   2> 217758 T892 C288 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217759 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@72eef1c[collection1] main{StandardDirectoryReader(segments_u:136:nrt _10(4.7):c1)}
   [junit4]   2> 217760 T892 C288 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {commit=} 0 5
   [junit4]   2> 217762 T892 oas.SolrTestCaseJ4.tearDown ###Ending testMultiThreaded
   [junit4]   2> 217766 T892 oas.SolrTestCaseJ4.setUp ###Starting testNonStringFieldsValues
   [junit4]   2> 217767 T892 C288 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 217768 T892 C288 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {deleteByQuery=*:*} 0 1
   [junit4]   2> 217770 T892 C288 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217770 T892 C288 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_u,generation=30}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_v,generation=31}
   [junit4]   2> 217771 T892 C288 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 31
   [junit4]   2> 217772 T892 C288 oass.SolrIndexSearcher.<init> Opening Searcher@58b6457b[collection1] main
   [junit4]   2> 217773 T892 C288 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217775 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@58b6457b[collection1] main{StandardDirectoryReader(segments_v:137:nrt)}
   [junit4]   2> 217775 T892 C288 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {commit=} 0 5
   [junit4]   2> 217779 T892 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 217782 T892 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217785 T892 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_v,generation=31}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_w,generation=32}
   [junit4]   2> 217785 T892 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 32
   [junit4]   2> 217787 T892 oass.SolrIndexSearcher.<init> Opening Searcher@dc6367a5[collection1] main
   [junit4]   2> 217788 T892 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217789 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@dc6367a5[collection1] main{StandardDirectoryReader(segments_w:139:nrt _11(4.7):c5/1:delGen=1)}
   [junit4]   2> 217790 T892 oas.SolrTestCaseJ4.tearDown ###Ending testNonStringFieldsValues
   [junit4]   2> 217795 T892 oas.SolrTestCaseJ4.setUp ###Starting testFailNonIndexedSigWithOverwriteDupes
   [junit4]   2> 217796 T892 C288 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 217797 T892 C288 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {deleteByQuery=*:*} 0 1
   [junit4]   2> 217798 T892 C288 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217799 T892 C288 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_w,generation=32}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_x,generation=33}
   [junit4]   2> 217800 T892 C288 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 33
   [junit4]   2> 217801 T892 C288 oass.SolrIndexSearcher.<init> Opening Searcher@6eb190a[collection1] main
   [junit4]   2> 217802 T892 C288 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217804 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6eb190a[collection1] main{StandardDirectoryReader(segments_x:140:nrt)}
   [junit4]   2> 217805 T892 C288 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {commit=} 0 7
   [junit4]   2> 217807 T892 oas.SolrTestCaseJ4.tearDown ###Ending testFailNonIndexedSigWithOverwriteDupes
   [junit4]   2> 217812 T892 oas.SolrTestCaseJ4.setUp ###Starting testDupeAllFieldsDetection
   [junit4]   2> 217812 T892 C288 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 217813 T892 C288 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {deleteByQuery=*:*} 0 1
   [junit4]   2> 217815 T892 C288 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217816 T892 C288 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_x,generation=33}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_y,generation=34}
   [junit4]   2> 217816 T892 C288 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 34
   [junit4]   2> 217817 T892 C288 oass.SolrIndexSearcher.<init> Opening Searcher@e464b687[collection1] main
   [junit4]   2> 217818 T892 C288 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217820 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@e464b687[collection1] main{StandardDirectoryReader(segments_y:141:nrt)}
   [junit4]   2> 217821 T892 C288 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {commit=} 0 6
   [junit4]   2> 217830 T892 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 217832 T892 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_y,generation=34}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1470124 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2553eb4d)),segFN=segments_z,generation=35}
   [junit4]   2> 217833 T892 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 35
   [junit4]   2> 217834 T892 oass.SolrIndexSearcher.<init> Opening Searcher@4035700d[collection1] main
   [junit4]   2> 217835 T892 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 217837 T894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4035700d[collection1] main{StandardDirectoryReader(segments_z:143:nrt _12(4.7):c3)}
   [junit4]   2> 217837 T892 oas.SolrTestCaseJ4.tearDown ###Ending testDupeAllFieldsDetection
   [junit4]   2> 217840 T892 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> 217840 T892 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=-1899279957
   [junit4]   2> 217840 T892 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@c1f8cbbb
   [junit4]   2> 217850 T892 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=42,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=253,cumulative_deletesById=0,cumulative_deletesByQuery=6,cumulative_errors=0}
   [junit4]   2> 217851 T892 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 217851 T892 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 217851 T892 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 217852 T892 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 217852 T892 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 217853 T892 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-SignatureUpdateProcessorFactoryTest-1391939118661/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-SignatureUpdateProcessorFactoryTest-1391939118661/index;done=false>>]
   [junit4]   2> 217853 T892 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-SignatureUpdateProcessorFactoryTest-1391939118661/index
   [junit4]   2> 217854 T892 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-SignatureUpdateProcessorFactoryTest-1391939118661 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-SignatureUpdateProcessorFactoryTest-1391939118661;done=false>>]
   [junit4]   2> 217854 T892 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-SignatureUpdateProcessorFactoryTest-1391939118661
   [junit4]   2> NOTE: test params are: codec=Asserting, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=tr_TR, timezone=Asia/Singapore
   [junit4]   2> NOTE: Linux 3.8.0-35-generic x86/IBM Corporation 1.7.0 (32-bit)/cpus=8,threads=1,free=103858696,total=175833088
   [junit4]   2> NOTE: All tests run in this JVM: [TestUpdate, SolrRequestParserTest, TestCodecSupport, SpellCheckCollatorTest, TestFastLRUCache, DisMaxRequestHandlerTest, FullSolrCloudDistribCmdsTest, HardAutoCommitTest, TestSchemaResource, PingRequestHandlerTest, TestDefaultSimilarityFactory, TestFastOutputStream, BasicZkTest, HighlighterTest, SuggesterTSTTest, CurrencyFieldOpenExchangeTest, CustomCollectionTest, RAMDirectoryFactoryTest, SignatureUpdateProcessorFactoryTest]
   [junit4] Completed on J1 in 2.29s, 6 tests, 1 failure <<< FAILURES!

[...truncated 444 lines...]
   [junit4] Suite: org.apache.solr.TestDistributedSearch
   [junit4]   2> 923311 T3582 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
   [junit4]   2> 923315 T3582 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> 923316 T3582 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 923323 T3582 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:34299
   [junit4]   2> 923324 T3582 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 923325 T3582 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 923325 T3582 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr
   [junit4]   2> 923326 T3582 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/'
   [junit4]   2> 923335 T3582 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/solr.xml
   [junit4]   2> 923373 T3582 oasc.CoreContainer.<init> New CoreConta

[...truncated too long message...]

2,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
   [junit4]   2> 1070746 T3955 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 1070747 T3955 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 1070748 T3955 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 1070749 T3955 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 1070750 T3955 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 1070750 T3955 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.handler.component.DistributedQueryElevationComponentTest-1391939972113/shard1/data [CachedDir<<refCount=0;path=./org.apache.solr.handler.component.DistributedQueryElevationComponentTest-1391939972113/shard1/data;done=false>>]
   [junit4]   2> 1070751 T3955 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.handler.component.DistributedQueryElevationComponentTest-1391939972113/shard1/data
   [junit4]   2> 1070751 T3955 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.handler.component.DistributedQueryElevationComponentTest-1391939972113/shard1/data/index [CachedDir<<refCount=0;path=./org.apache.solr.handler.component.DistributedQueryElevationComponentTest-1391939972113/shard1/data/index;done=false>>]
   [junit4]   2> 1070752 T3955 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.handler.component.DistributedQueryElevationComponentTest-1391939972113/shard1/data/index
   [junit4]   2> 1070753 T3955 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/ad/rt,null}
   [junit4]   2> 1070826 T3955 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=65069406
   [junit4]   2> 1070827 T3955 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@fa19c24e
   [junit4]   2> 1070838 T3955 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=3,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
   [junit4]   2> 1070839 T3955 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 1070841 T3955 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 1070842 T3955 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 1070844 T3955 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 1070845 T3955 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 1070846 T3955 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.handler.component.DistributedQueryElevationComponentTest-1391939972113/shard2/data [CachedDir<<refCount=0;path=./org.apache.solr.handler.component.DistributedQueryElevationComponentTest-1391939972113/shard2/data;done=false>>]
   [junit4]   2> 1070848 T3955 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.handler.component.DistributedQueryElevationComponentTest-1391939972113/shard2/data
   [junit4]   2> 1070848 T3955 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.handler.component.DistributedQueryElevationComponentTest-1391939972113/shard2/data/index [CachedDir<<refCount=0;path=./org.apache.solr.handler.component.DistributedQueryElevationComponentTest-1391939972113/shard2/data/index;done=false>>]
   [junit4]   2> 1070849 T3955 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.handler.component.DistributedQueryElevationComponentTest-1391939972113/shard2/data/index
   [junit4]   2> 1070850 T3955 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/ad/rt,null}
   [junit4]   2> 1070902 T3955 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=DistributedQueryElevationComponentTest -Dtests.method=testDistribSearch -Dtests.seed=2696C0E878F27C17 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=kn_IN -Dtests.timezone=America/Sitka -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 1.87s J1 | DistributedQueryElevationComponentTest.testDistribSearch <<<
   [junit4]    > Throwable #1: junit.framework.AssertionFailedError: .responseHeader.params.version!=rows (unordered or missing)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([2696C0E878F27C17:A7704EF00FAD1C2B]:0)
   [junit4]    > 	at junit.framework.Assert.fail(Assert.java:50)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.compareSolrResponses(BaseDistributedSearchTestCase.java:840)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.compareResponses(BaseDistributedSearchTestCase.java:859)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:562)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:542)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:521)
   [junit4]    > 	at org.apache.solr.handler.component.DistributedQueryElevationComponentTest.doTest(DistributedQueryElevationComponentTest.java:81)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:867)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:780)
   [junit4]   2> 1070914 T3955 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: test params are: codec=Lucene42, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=kn_IN, timezone=America/Sitka
   [junit4]   2> NOTE: Linux 3.8.0-35-generic x86/IBM Corporation 1.7.0 (32-bit)/cpus=8,threads=1,free=133965896,total=268435456
   [junit4]   2> NOTE: All tests run in this JVM: [TestUpdate, SolrRequestParserTest, TestCodecSupport, SpellCheckCollatorTest, TestFastLRUCache, DisMaxRequestHandlerTest, FullSolrCloudDistribCmdsTest, HardAutoCommitTest, TestSchemaResource, PingRequestHandlerTest, TestDefaultSimilarityFactory, TestFastOutputStream, BasicZkTest, HighlighterTest, SuggesterTSTTest, CurrencyFieldOpenExchangeTest, CustomCollectionTest, RAMDirectoryFactoryTest, SignatureUpdateProcessorFactoryTest, ShardRoutingCustomTest, TestStressReorder, BadIndexSchemaTest, TestSolrDeletionPolicy2, HdfsDirectoryTest, AlternateDirectoryTest, ParsingFieldUpdateProcessorsTest, SpellingQueryConverterTest, TestPhraseSuggestions, TestNonNRTOpen, TestCSVLoader, SolrIndexConfigTest, CollectionsAPIDistributedZkTest, TestInfoStreamLogging, DebugComponentTest, TestBinaryResponseWriter, TestAtomicUpdateErrorCases, PluginInfoTest, TestQuerySenderListener, ZkNodePropsTest, TestFieldTypeCollectionResource, NumericFieldsTest, StandardRequestHandlerTest, TestSolrXml, TestCoreDiscovery, ShardSplitTest, TestAnalyzedSuggestions, AnalysisErrorHandlingTest, SolrCoreCheckLockOnStartupTest, RequestHandlersTest, NoCacheHeaderTest, OpenCloseCoreStressTest, DateFieldTest, TestPartialUpdateDeduplication, DocValuesMissingTest, TestComponentsName, DistanceFunctionTest, MinimalSchemaTest, TestStressRecovery, TestCollapseQParserPlugin, SolrXmlInZkTest, SuggesterWFSTTest, TestRandomFaceting, TestFastWriter, TestIndexingPerformance, PreAnalyzedUpdateProcessorTest, TestMaxScoreQueryParser, TestIBSimilarityFactory, TestRTGBase, CoreMergeIndexesAdminHandlerTest, TestCopyFieldCollectionResource, BlockDirectoryTest, CurrencyFieldXmlFileTest, TestQuerySenderNoQuery, ChaosMonkeySafeLeaderTest, TestSolrXmlPersistence, OutputWriterTest, XmlUpdateRequestHandlerTest, IndexBasedSpellCheckerTest, TestSolr4Spatial, TestReload, HdfsBasicDistributedZkTest, ConvertedLegacyTest, TestShardHandlerFactory, TestQueryTypes, TestSolrJ, DirectSolrConnectionTest, NotRequiredUniqueKeyTest, SimplePostToolTest, ZkCLITest, SliceStateUpdateTest, TestDistributedSearch, TestDistributedGrouping, TestHashPartitioner, TestRealTimeGet, TestMultiCoreConfBootstrap, TestRangeQuery, SimpleFacetsTest, StatsComponentTest, TestGroupingSearch, PeerSyncTest, TestFiltering, DistributedQueryElevationComponentTest]
   [junit4] Completed on J1 in 1.91s, 1 test, 1 failure <<< FAILURES!

[...truncated 450 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:459: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:452: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:491: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1308: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:941: There were test failures: 360 suites, 1558 tests, 4 failures, 35 ignored (7 assumptions)

Total time: 55 minutes 3 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/ibm-j9-jdk7 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-4.x-Linux (64bit/jdk1.7.0_60-ea-b04) - Build # 9311 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/9311/
Java: 64bit/jdk1.7.0_60-ea-b04 -XX:+UseCompressedOops -XX:+UseSerialGC

1 tests failed.
REGRESSION:  org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testDistribSearch

Error Message:
No live SolrServers available to handle this request:[http://127.0.0.1:43912, http://127.0.0.1:53218, http://127.0.0.1:42392, http://127.0.0.1:42825, http://127.0.0.1:52432]

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:43912, http://127.0.0.1:53218, http://127.0.0.1:42392, http://127.0.0.1:42825, http://127.0.0.1:52432]
	at __randomizedtesting.SeedInfo.seed([30A7063A7A1B7867:B14188220D44185B]:0)
	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:640)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1623)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1649)
	at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testIndexingBatchPerRequestWithHttpSolrServer(FullSolrCloudDistribCmdsTest.java:353)
	at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.doTest(FullSolrCloudDistribCmdsTest.java:143)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:867)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:744)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server Error

request: http://127.0.0.1:52432/admin/collections?action=CREATE&numShards=7&replicationFactor=3&maxShardsPerNode=100000&collection.configName=conf1&name=collection2&wt=javabin&version=2
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:495)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:199)
	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:283)
	... 46 more




Build Log:
[...truncated 10511 lines...]
   [junit4] Suite: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
   [junit4]   2> 672495 T3284 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
   [junit4]   2> 672498 T3284 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-FullSolrCloudDistribCmdsTest-1391945950284
   [junit4]   2> 672498 T3284 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 672499 T3285 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 672599 T3284 oasc.ZkTestServer.run start zk server on port:36694
   [junit4]   2> 672600 T3284 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 672768 T3291 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4adc9f28 name:ZooKeeperConnection Watcher:127.0.0.1:36694 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 672768 T3284 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 672769 T3284 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 672773 T3284 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 672774 T3293 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2aa8fb67 name:ZooKeeperConnection Watcher:127.0.0.1:36694/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 672775 T3284 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 672775 T3284 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 672778 T3284 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 672781 T3284 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 672782 T3284 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 672784 T3284 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 672784 T3284 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 672787 T3284 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 672787 T3284 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 672789 T3284 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 672790 T3284 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 672791 T3284 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 672792 T3284 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 672794 T3284 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 672794 T3284 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 672796 T3284 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 672796 T3284 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 672798 T3284 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 672798 T3284 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 672800 T3284 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 672801 T3284 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 672803 T3284 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 672803 T3284 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 672805 T3284 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 672805 T3284 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 672807 T3284 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 672807 T3284 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 672870 T3284 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 672873 T3284 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:42825
   [junit4]   2> 672874 T3284 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 672874 T3284 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 672874 T3284 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1391945950596
   [junit4]   2> 672874 T3284 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1391945950596/'
   [junit4]   2> 672888 T3284 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1391945950596/solr.xml
   [junit4]   2> 672907 T3284 oasc.CoreContainer.<init> New CoreContainer 902895666
   [junit4]   2> 672908 T3284 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1391945950596/]
   [junit4]   2> 672908 T3284 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 672908 T3284 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 672909 T3284 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 672909 T3284 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 672909 T3284 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 672909 T3284 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 672909 T3284 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 672910 T3284 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 672910 T3284 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 672912 T3284 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 672912 T3284 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 672912 T3284 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 672913 T3284 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:36694/solr
   [junit4]   2> 672913 T3284 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 672913 T3284 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 672916 T3304 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6a335074 name:ZooKeeperConnection Watcher:127.0.0.1:36694 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 672916 T3284 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 672918 T3284 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 672919 T3306 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5573990b name:ZooKeeperConnection Watcher:127.0.0.1:36694/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 672919 T3284 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 672920 T3284 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 672923 T3284 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 672925 T3284 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 672926 T3284 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42825_
   [junit4]   2> 672927 T3284 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:42825_
   [junit4]   2> 672929 T3284 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 672930 T3284 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 672942 T3284 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:42825_
   [junit4]   2> 672942 T3284 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 672944 T3284 oasc.Overseer.start Overseer (id=91222569798074371-127.0.0.1:42825_-n_0000000000) starting
   [junit4]   2> 672946 T3284 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 672948 T3308 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 672949 T3284 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 672949 T3308 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
   [junit4]   2> 672950 T3284 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 672952 T3284 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 672953 T3307 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 672955 T3309 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 672956 T3309 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 672959 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 672959 T3309 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 672960 T3307 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 672961 T3307 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:42825",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:42825_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 672961 T3307 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
   [junit4]   2> 672961 T3307 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 672962 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 672963 T3306 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 673959 T3309 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 673960 T3309 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1391945950596/collection1
   [junit4]   2> 673960 T3309 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
   [junit4]   2> 673961 T3309 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 673961 T3309 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 673962 T3309 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 673962 T3309 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1391945950596/collection1/'
   [junit4]   2> 673962 T3309 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1391945950596/collection1/lib/classes/' to classloader
   [junit4]   2> 673963 T3309 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1391945950596/collection1/lib/README' to classloader
   [junit4]   2> 673984 T3309 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 674003 T3309 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 674005 T3309 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 674010 T3309 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 674184 T3309 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 674186 T3309 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 674194 T3309 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 674197 T3309 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 674215 T3309 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 674215 T3309 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1391945950596/collection1/, dataDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1391945950284/control/data/
   [junit4]   2> 674216 T3309 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2e62ff34
   [junit4]   2> 674216 T3309 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1391945950284/control/data
   [junit4]   2> 674216 T3309 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1391945950284/control/data/index/
   [junit4]   2> 674216 T3309 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1391945950284/control/data/index' doesn't exist. Creating new index...
   [junit4]   2> 674217 T3309 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1391945950284/control/data/index
   [junit4]   2> 674217 T3309 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=14, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7705605394439433]
   [junit4]   2> 674217 T3309 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@2e88398a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7cf06eb9),segFN=segments_1,generation=1}
   [junit4]   2> 674218 T3309 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 674220 T3309 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 674220 T3309 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 674220 T3309 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 674220 T3309 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 674220 T3309 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 674221 T3309 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 674221 T3309 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 674221 T3309 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 674221 T3309 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 674222 T3309 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 674222 T3309 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 674222 T3309 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 674222 T3309 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 674222 T3309 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 674223 T3309 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 674223 T3309 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 674225 T3309 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 674227 T3309 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 674227 T3309 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 674227 T3309 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=15, maxMergeAtOnceExplicit=30, maxMergedSegmentMB=79.0693359375, floorSegmentMB=1.1513671875, forceMergeDeletesPctAllowed=12.415228739741067, segmentsPerTier=50.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 674228 T3309 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@2e88398a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7cf06eb9),segFN=segments_1,generation=1}
   [junit4]   2> 674228 T3309 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 674228 T3309 oass.SolrIndexSearcher.<init> Opening Searcher@57268c25[collection1] main
   [junit4]   2> 674230 T3310 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@57268c25[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 674230 T3309 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 674231 T3284 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 674231 T3284 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 674231 T3313 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:42825 collection:control_collection shard:shard1
   [junit4]   2> 674232 T3284 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 674232 T3313 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 674233 T3315 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@63b35223 name:ZooKeeperConnection Watcher:127.0.0.1:36694/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 674234 T3284 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 674235 T3284 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 674237 T3284 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 674238 T3313 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 674240 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 674240 T3313 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 674240 T3313 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C2455 name=collection1 org.apache.solr.core.SolrCore@3baf3c8a url=http://127.0.0.1:42825/collection1 node=127.0.0.1:42825_ C2455_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:42825, core=collection1, node_name=127.0.0.1:42825_}
   [junit4]   2> 674241 T3313 C2455 P42825 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:42825/collection1/
   [junit4]   2> 674241 T3313 C2455 P42825 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 674241 T3313 C2455 P42825 oasc.SyncStrategy.syncToMe http://127.0.0.1:42825/collection1/ has no replicas
   [junit4]   2> 674241 T3313 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:42825/collection1/ shard1
   [junit4]   2> 674241 T3313 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 674242 T3307 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 674245 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 674247 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 674251 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 674308 T3284 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
   [junit4]   2> 674308 T3284 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 674310 T3284 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:48555
   [junit4]   2> 674310 T3284 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 674311 T3284 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 674311 T3284 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1391945952024
   [junit4]   2> 674311 T3284 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1391945952024/'
   [junit4]   2> 674321 T3284 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1391945952024/solr.xml
   [junit4]   2> 674340 T3284 oasc.CoreContainer.<init> New CoreContainer 1101670486
   [junit4]   2> 674341 T3284 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1391945952024/]
   [junit4]   2> 674341 T3284 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 674342 T3284 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 674342 T3284 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 674342 T3284 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 674342 T3284 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 674343 T3284 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 674343 T3284 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 674343 T3284 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 674343 T3284 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 674345 T3284 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 674345 T3284 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 674346 T3284 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 674346 T3284 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:36694/solr
   [junit4]   2> 674346 T3284 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 674347 T3284 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 674348 T3326 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a3e47c8 name:ZooKeeperConnection Watcher:127.0.0.1:36694 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 674349 T3284 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 674351 T3284 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 674352 T3328 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@472d6ce3 name:ZooKeeperConnection Watcher:127.0.0.1:36694/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 674352 T3284 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 674354 T3315 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 674354 T3306 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 674360 T3284 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 674398 T3313 oasc.ZkController.register We are http://127.0.0.1:42825/collection1/ and leader is http://127.0.0.1:42825/collection1/
   [junit4]   2> 674399 T3313 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:42825
   [junit4]   2> 674399 T3313 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 674399 T3313 oasc.ZkController.publish publishing core=collection1 state=active
   [junit4]   2> 674399 T3313 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 674401 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 674401 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 674401 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 674401 T3313 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 674402 T3307 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 674403 T3307 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:42825",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:42825_",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 674405 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 674506 T3315 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 674506 T3306 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 674507 T3328 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 675363 T3284 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48555_
   [junit4]   2> 675365 T3284 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:48555_
   [junit4]   2> 675367 T3315 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 675367 T3306 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 675367 T3328 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 675372 T3329 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 675372 T3329 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 675373 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 675373 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 675373 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 675373 T3329 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 675374 T3307 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 675374 T3307 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:48555",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:48555_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 675375 T3307 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3]
   [junit4]   2> 675375 T3307 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 675376 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 675557 T3315 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 675557 T3328 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 675558 T3306 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 676373 T3329 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 676374 T3329 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1391945952024/collection1
   [junit4]   2> 676374 T3329 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 676375 T3329 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 676375 T3329 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 676376 T3329 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 676376 T3329 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1391945952024/collection1/'
   [junit4]   2> 676376 T3329 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1391945952024/collection1/lib/classes/' to classloader
   [junit4]   2> 676377 T3329 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1391945952024/collection1/lib/README' to classloader
   [junit4]   2> 676399 T3329 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 676427 T3329 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 676429 T3329 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 676434 T3329 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 676613 T3329 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 676615 T3329 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 676616 T3329 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 676619 T3329 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 676638 T3329 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 676638 T3329 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1391945952024/collection1/, dataDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1391945950284/jetty1/
   [junit4]   2> 676638 T3329 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2e62ff34
   [junit4]   2> 676639 T3329 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1391945950284/jetty1
   [junit4]   2> 676639 T3329 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1391945950284/jetty1/index/
   [junit4]   2> 676639 T3329 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1391945950284/jetty1/index' doesn't exist. Creating new index...
   [junit4]   2> 676639 T3329 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1391945950284/jetty1/index
   [junit4]   2> 676639 T3329 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=14, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7705605394439433]
   [junit4]   2> 676640 T3329 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@6eefd6d5 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@73ba4860),segFN=segments_1,generation=1}
   [junit4]   2> 676640 T3329 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 676643 T3329 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 676643 T3329 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 676643 T3329 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 676643 T3329 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 676643 T3329 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 676644 T3329 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 676644 T3329 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 676644 T3329 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 676644 T3329 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 676645 T3329 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 676645 T3329 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 676645 T3329 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 676645 T3329 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 676646 T3329 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 676646 T3329 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 676646 T3329 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 676648 T3329 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 676650 T3329 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 676650 T3329 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 676651 T3329 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=15, maxMergeAtOnceExplicit=30, maxMergedSegmentMB=79.0693359375, floorSegmentMB=1.1513671875, forceMergeDeletesPctAllowed=12.415228739741067, segmentsPerTier=50.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 676651 T3329 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@6eefd6d5 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@73ba4860),segFN=segments_1,generation=1}
   [junit4]   2> 676651 T3329 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 676651 T3329 oass.SolrIndexSearcher.<init> Opening Searcher@3b233ff2[collection1] main
   [junit4]   2> 676654 T3330 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3b233ff2[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 676655 T3329 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 676656 T3284 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 676656 T3284 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 676656 T3333 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:48555 collection:collection1 shard:shard1
   [junit4]   2> 676658 T3333 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 676663 T3333 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 676665 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 676665 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 676665 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 676665 T3333 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 676666 T3333 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C2456 name=collection1 org.apache.solr.core.SolrCore@704d845 url=http://127.0.0.1:48555/collection1 node=127.0.0.1:48555_ C2456_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:48555, core=collection1, node_name=127.0.0.1:48555_}
   [junit4]   2> 676666 T3333 C2456 P48555 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:48555/collection1/
   [junit4]   2> 676666 T3333 C2456 P48555 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 676666 T3333 C2456 P48555 oasc.SyncStrategy.syncToMe http://127.0.0.1:48555/collection1/ has no replicas
   [junit4]   2> 676666 T3333 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:48555/collection1/ shard1
   [junit4]   2> 676666 T3333 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 676667 T3307 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 676669 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 676670 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 676672 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 676727 T3284 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
   [junit4]   2> 676727 T3284 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 676729 T3284 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:53218
   [junit4]   2> 676730 T3284 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 676730 T3284 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 676730 T3284 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1391945954443
   [junit4]   2> 676731 T3284 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1391945954443/'
   [junit4]   2> 676740 T3284 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1391945954443/solr.xml
   [junit4]   2> 676774 T3284 oasc.CoreContainer.<init> New CoreContainer 350968084
   [junit4]   2> 676775 T3284 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1391945954443/]
   [junit4]   2> 676775 T3284 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 676775 T3284 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 676776 T3284 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 676776 T3306 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 676776 T3315 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 676776 T3328 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 676776 T3284 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 676777 T3284 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 676777 T3284 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 676777 T3284 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 676777 T3284 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 676777 T3284 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 676779 T3284 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 676780 T3284 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 676780 T3284 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 676780 T3284 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:36694/solr
   [junit4]   2> 676780 T3284 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 676781 T3284 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 676782 T3344 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3d35d211 name:ZooKeeperConnection Watcher:127.0.0.1:36694 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 676782 T3284 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 676784 T3284 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 676785 T3346 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2c5e8d5d name:ZooKeeperConnection Watcher:127.0.0.1:36694/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 676786 T3284 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 676789 T3284 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 676821 T3333 oasc.ZkController.register We are http://127.0.0.1:48555/collection1/ and leader is http://127.0.0.1:48555/collection1/
   [junit4]   2> 676821 T3333 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:48555
   [junit4]   2> 676821 T3333 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 676821 T3333 oasc.ZkController.publish publishing core=collection1 state=active
   [junit4]   2> 676821 T3333 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 676822 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 676823 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 676823 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 676823 T3333 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 676824 T3307 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 676825 T3307 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:48555",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:48555_",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 676827 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 676929 T3306 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 676929 T3315 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 676929 T3346 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 676929 T3328 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 677791 T3284 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53218_
   [junit4]   2> 677793 T3284 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:53218_
   [junit4]   2> 677795 T3315 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 677795 T3328 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 677795 T3346 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 677795 T3306 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 677799 T3347 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 677799 T3347 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 677800 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 677800 T3347 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 677800 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 677801 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 677802 T3307 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 677803 T3307 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:53218",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:53218_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 677803 T3307 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
   [junit4]   2> 677803 T3307 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 677805 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 677927 T3328 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 677928 T3315 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 677928 T3306 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 677928 T3346 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 678801 T3347 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 678801 T3347 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1391945954443/collection1
   [junit4]   2> 678801 T3347 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 678802 T3347 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 678802 T3347 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 678803 T3347 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 678803 T3347 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1391945954443/collection1/'
   [junit4]   2> 678804 T3347 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1391945954443/collection1/lib/classes/' to classloader
   [junit4]   2> 678804 T3347 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1391945954443/collection1/lib/README' to classloader
   [junit4]   2> 678826 T3347 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 678845 T3347 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 678847 T3347 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 678852 T3347 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 679047 T3347 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 679048 T3347 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 679049 T3347 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 679052 T3347 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 679072 T3347 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 679072 T3347 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1391945954443/collection1/, dataDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1391945950284/jetty2/
   [junit4]   2> 679073 T3347 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2e62ff34
   [junit4]   2> 679073 T3347 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1391945950284/jetty2
   [junit4]   2> 679073 T3347 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1391945950284/jetty2/index/
   [junit4]   2> 679073 T3347 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1391945950284/jetty2/index' doesn't exist. Creating new index...
   [junit4]   2> 679074 T3347 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1391945950284/jetty2/index
   [junit4]   2> 679074 T3347 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=14, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7705605394439433]
   [junit4]   2> 679074 T3347 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@62318bb7 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4fe17257),segFN=segments_1,generation=1}
   [junit4]   2> 679075 T3347 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 679077 T3347 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 679078 T3347 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 679078 T3347 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 679078 T3347 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 679078 T3347 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 679078 T3347 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 679079 T3347 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 679079 T3347 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 679079 T3347 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 679079 T3347 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 679080 T3347 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 679080 T3347 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 679080 T3347 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 679080 T3347 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 679081 T3347 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 679081 T3347 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 679083 T3347 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 679085 T3347 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 679085 T3347 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 679085 T3347 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=15, maxMergeAtOnceExplicit=30, maxMergedSegmentMB=79.0693359375, floorSegmentMB=1.1513671875, forceMergeDeletesPctAllowed=12.415228739741067, segmentsPerTier=50.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 679086 T3347 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@62318bb7 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4fe17257),segFN=segments_1,generation=1}
   [junit4]   2> 679086 T3347 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 679086 T3347 oass.SolrIndexSearcher.<init> Opening Searcher@1a421b3b[collection1] main
   [junit4]   2> 679089 T3348 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1a421b3b[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 679091 T3347 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 679091 T3284 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 679091 T3284 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 679091 T3351 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:53218 collection:collection1 shard:shard2
   [junit4]   2> 679093 T3351 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
   [junit4]   2> 679097 T3351 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
   [junit4]   2> 679099 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 679099 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 679099 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 679099 T3351 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 679099 T3351 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C2457 name=collection1 org.apache.solr.core.SolrCore@21d2a335 url=http://127.0.0.1:53218/collection1 node=127.0.0.1:53218_ C2457_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:53218, core=collection1, node_name=127.0.0.1:53218_}
   [junit4]   2> 679100 T3351 C2457 P53218 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:53218/collection1/
   [junit4]   2> 679100 T3351 C2457 P53218 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 679100 T3351 C2457 P53218 oasc.SyncStrategy.syncToMe http://127.0.0.1:53218/collection1/ has no replicas
   [junit4]   2> 679100 T3351 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:53218/collection1/ shard2
   [junit4]   2> 679100 T3351 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
   [junit4]   2> 679100 T3307 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 679104 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 679106 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 679110 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 679162 T3284 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
   [junit4]   2> 679163 T3284 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 679165 T3284 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:42392
   [junit4]   2> 679165 T3284 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 679166 T3284 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 679166 T3284 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1391945956878
   [junit4]   2> 679166 T3284 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1391945956878/'
   [junit4]   2> 679176 T3284 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1391945956878/solr.xml
   [junit4]   2> 679196 T3284 oasc.CoreContainer.<init> New CoreContainer 1070111183
   [junit4]   2> 679197 T3284 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1391945956878/]
   [junit4]   2> 679197 T3284 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 679197 T3284 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 679198 T3284 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 679198 T3284 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 679198 T3284 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 679198 T3284 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 679198 T3284 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 679199 T3284 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 679199 T3284 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 679201 T3284 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 679201 T3284 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 679201 T3284 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 679201 T3284 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:36694/solr
   [junit4]   2> 679202 T3284 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 679202 T3284 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 679204 T3362 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@aea05c0 name:ZooKeeperConnection Watcher:127.0.0.1:36694 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 679204 T3284 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 679221 T3284 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 679227 T3328 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 679228 T3315 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 679228 T3306 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 679227 T3346 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 679230 T3364 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@69ae484d name:ZooKeeperConnection Watcher:127.0.0.1:36694/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 679230 T3284 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 679234 T3284 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 679256 T3351 oasc.ZkController.register We are http://127.0.0.1:53218/collection1/ and leader is http://127.0.0.1:53218/collection1/
   [junit4]   2> 679257 T3351 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:53218
   [junit4]   2> 679257 T3351 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 679257 T3351 oasc.ZkController.publish publishing core=collection1 state=active
   [junit4]   2> 679257 T3351 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 679259 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 679259 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 679259 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 679259 T3351 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 679261 T3307 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 679261 T3307 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:53218",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:53218_",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "core_node_name":"core_node2"}
   [junit4]   2> 679263 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 679365 T3315 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 679365 T3364 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 679365 T3328 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 679365 T3346 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 679365 T3306 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 680236 T3284 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42392_
   [junit4]   2> 680238 T3284 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:42392_
   [junit4]   2> 680241 T3346 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 680241 T3328 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 680241 T3306 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 680241 T3364 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 680241 T3315 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 680248 T3365 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 680248 T3365 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 680277 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 680278 T3365 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 680278 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 680278 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 680279 T3307 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 680280 T3307 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:42392",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:42392_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 680280 T3307 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
   [junit4]   2> 680280 T3307 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
   [junit4]   2> 680283 T3306 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 680385 T3364 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 680385 T3328 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 680385 T3315 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 680385 T3306 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 680385 T3346 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 681278 T3365 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 681278 T3365 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1391945956878/collection1
   [junit4]   2> 681278 T3365 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 681279 T3365 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 681279 T3365 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 681280 T3365 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 681281 T3365 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1391945956878/collection1/'
   [junit4]   2> 681281 T3365 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1391945956878/collection1/lib/classes/' to classloader
   [junit4]   2> 681282 T3365 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1391945956878/collection1/lib/README' to classloader
   [junit4]   2> 681328 T3365 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 681357 T3365 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 681358 T3365 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 681365 T3365 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 681552 T3365 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 681554 T3365 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 681555 T3365 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 681558 T3365 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 681576 T3365 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 681576 T3365 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1391945956878/collection1/, dataDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1391945950284/jetty3/
   [junit4]   2> 681576 T3365 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2e62ff34
   [junit4]   2> 681577 T3365 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1391945950284/jetty3
   [junit4]   2> 681577 T3365 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1391945950284/jetty3/index/
   [junit4]   2> 681577 T3365 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1391945950284/jetty3/index' doesn't exist. Creating new index...
   [junit4]   2> 681577 T3365 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1391945950284/jetty3/index
   [junit4]   2> 681578 T3365 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=14, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7705605394439433]
   [junit4]   2> 681578 T3365 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@5d64f67e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3ad55623),segFN=segments_1,generation=1}
   [junit4]   2> 681578 T3365 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
 

[...truncated too long message...]

uster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 731999 T3476 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 732007 T3284 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 42392
   [junit4]   2> 732007 T3284 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1070111183
   [junit4]   2> 732011 T3401 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 732012 T3420 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 732012 T3401 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:52432_
   [junit4]   2> 732012 T3401 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 732014 T3401 oasc.Overseer.start Overseer (id=91222569798074382-127.0.0.1:52432_-n_0000000005) starting
   [junit4]   2> 732018 T3479 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 732018 T3479 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
   [junit4]   2> 732018 T3478 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 732032 T3284 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
   [junit4]   2> 732084 T3284 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 49007
   [junit4]   2> 732085 T3284 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 52432
   [junit4]   2> 732085 T3284 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1699353417
   [junit4]   2> 732089 T3420 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:43912_
   [junit4]   2> 732090 T3420 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 732092 T3420 oasc.Overseer.start Overseer (id=91222569798074384-127.0.0.1:43912_-n_0000000006) starting
   [junit4]   2> 732096 T3481 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 732096 T3420 oascc.ZkStateReader$3.process Updating live nodes... (1)
   [junit4]   2> 732097 T3481 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
   [junit4]   2> 732098 T3480 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 732112 T3284 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
   [junit4]   2> 732163 T3284 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 43912
   [junit4]   2> 732163 T3284 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1084058489
   [junit4]   2> 732187 T3284 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
   [junit4]   2> 732246 T3284 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
   [junit4]   2> 732248 T3284 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:36694 36694
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=FullSolrCloudDistribCmdsTest -Dtests.method=testDistribSearch -Dtests.seed=30A7063A7A1B7867 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_PR -Dtests.timezone=America/Argentina/Tucuman -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   59.9s J1 | FullSolrCloudDistribCmdsTest.testDistribSearch <<<
   [junit4]    > Throwable #1: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:43912, http://127.0.0.1:53218, http://127.0.0.1:42392, http://127.0.0.1:42825, http://127.0.0.1:52432]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([30A7063A7A1B7867:B14188220D44185B]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:352)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:640)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1623)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1649)
   [junit4]    > 	at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testIndexingBatchPerRequestWithHttpSolrServer(FullSolrCloudDistribCmdsTest.java:353)
   [junit4]    > 	at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.doTest(FullSolrCloudDistribCmdsTest.java:143)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:867)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:744)
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server Error
   [junit4]    > request: http://127.0.0.1:52432/admin/collections?action=CREATE&numShards=7&replicationFactor=3&maxShardsPerNode=100000&collection.configName=conf1&name=collection2&wt=javabin&version=2
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:495)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:199)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:283)
   [junit4]    > 	... 46 more
   [junit4]   2> 732385 T3284 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> 59895 T3283 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 5 leaked thread(s).
   [junit4]   2> 733433 T3346 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 733511 T3364 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 733588 T3401 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 733666 T3420 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> NOTE: test params are: codec=Appending, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=es_PR, timezone=America/Argentina/Tucuman
   [junit4]   2> NOTE: Linux 3.8.0-35-generic amd64/Oracle Corporation 1.7.0_60-ea (64-bit)/cpus=8,threads=1,free=108943760,total=240222208
   [junit4]   2> NOTE: All tests run in this JVM: [StandardRequestHandlerTest, AlternateDirectoryTest, TestNumberUtils, SpellingQueryConverterTest, DirectUpdateHandlerTest, SignatureUpdateProcessorFactoryTest, TestRandomFaceting, TestInfoStreamLogging, TestHashPartitioner, TestCopyFieldCollectionResource, DisMaxRequestHandlerTest, TestDistributedMissingSort, StatsComponentTest, ChaosMonkeyNothingIsSafeTest, OutputWriterTest, ConnectionManagerTest, TestSolrCoreProperties, TestWordDelimiterFilterFactory, TestComponentsName, UniqFieldsUpdateProcessorFactoryTest, DebugComponentTest, DistributedDebugComponentTest, CurrencyFieldOpenExchangeTest, TimeZoneUtilsTest, CoreContainerCoreInitFailuresTest, TestFreeTextSuggestions, TestCoreDiscovery, TestSolrJ, SuggesterTSTTest, TestSolrXmlPersistor, TestReloadAndDeleteDocs, TestManagedSchemaFieldResource, TestStressUserVersions, BlockCacheTest, TestAtomicUpdateErrorCases, TestFastWriter, DistanceFunctionTest, TestSolrDeletionPolicy2, ConvertedLegacyTest, TestFuzzyAnalyzedSuggestions, TestUtils, TestQuerySenderListener, TestCodecSupport, HdfsUnloadDistributedZkTest, AssignTest, JSONWriterTest, TestSchemaSimilarityResource, OpenCloseCoreStressTest, LeaderElectionIntegrationTest, TestFoldingMultitermQuery, DistributedQueryComponentCustomSortTest, TestDistribDocBasedVersion, MultiTermTest, TestPartialUpdateDeduplication, FieldAnalysisRequestHandlerTest, TestShardHandlerFactory, BadIndexSchemaTest, TestUniqueKeyFieldResource, TestRecoveryHdfs, HdfsSyncSliceTest, SolrRequestParserTest, PreAnalyzedUpdateProcessorTest, TestOmitPositions, SpellCheckCollatorTest, TestClassNameShortening, TestZkChroot, HighlighterTest, ChaosMonkeySafeLeaderTest, TestBinaryResponseWriter, TestIndexingPerformance, TestArbitraryIndexDir, TestCSVLoader, TestAnalyzedSuggestions, TermVectorComponentDistributedTest, TestReload, DirectSolrSpellCheckerTest, TestCloudManagedSchema, TestUpdate, SuggesterWFSTTest, FileUtilsTest, MinimalSchemaTest, TestFastLRUCache, TestSweetSpotSimilarityFactory, PreAnalyzedFieldTest, TestMultiCoreConfBootstrap, TestSolrXmlPersistence, SortByFunctionTest, BasicDistributedZkTest, BasicDistributedZk2Test, ShardRoutingTest, BasicZkTest, FullSolrCloudDistribCmdsTest]
   [junit4] Completed on J1 in 61.42s, 1 test, 1 error <<< FAILURES!

[...truncated 534 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:459: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:439: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:491: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1308: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:941: There were test failures: 360 suites, 1558 tests, 1 error, 35 ignored (7 assumptions)

Total time: 43 minutes 7 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0_60-ea-b04 -XX:+UseCompressedOops -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure