You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2016/11/25 09:17:08 UTC

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 969 - Unstable!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/969/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC

4 tests failed.
FAILED:  org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testErrorHandling

Error Message:
unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes.    Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>

Stack Trace:
org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
  Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
	at __randomizedtesting.SeedInfo.seed([1E6F7E32F1994784:2B9B000CA74FC650]:0)
	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testErrorHandling(UIMAUpdateRequestProcessorTest.java:137)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testTwoUpdates

Error Message:
unknown UpdateRequestProcessorChain: <add><doc><field name="id">1</field><field name="text">The Apache Software Foundation is happy to announce BarCampApache Sydney, Australia, the first ASF-backed event in the Southern Hemisphere!</field></doc></add>

Stack Trace:
org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">1</field><field name="text">The Apache Software Foundation is happy to announce BarCampApache Sydney, Australia, the first ASF-backed event in the Southern Hemisphere!</field></doc></add>
	at __randomizedtesting.SeedInfo.seed([1E6F7E32F1994784:1325296DE9B6252A]:0)
	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testTwoUpdates(UIMAUpdateRequestProcessorTest.java:101)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing

Error Message:
unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes.    Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>

Stack Trace:
org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
  Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
	at __randomizedtesting.SeedInfo.seed([1E6F7E32F1994784:DE17A40B66AC62E]:0)
	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing(UIMAUpdateRequestProcessorTest.java:83)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testMultiplierProcessing

Error Message:
unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes.    Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>

Stack Trace:
org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
  Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
	at __randomizedtesting.SeedInfo.seed([1E6F7E32F1994784:256EB49FF6CD6B70]:0)
	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing(UIMAUpdateRequestProcessorTest.java:83)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testMultiplierProcessing(UIMAUpdateRequestProcessorTest.java:177)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 23814 lines...]
   [junit4] Suite: org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/contrib/solr-uima/test/J0/temp/solr.uima.processor.UIMAUpdateRequestProcessorTest_1E6F7E32F1994784-001/init-core-data-001
   [junit4]   2> log4j:WARN No appenders could be found for logger (org.eclipse.jetty.util.log).
   [junit4]   2> log4j:WARN Please initialize the log4j system properly.
   [junit4]   2> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
   [junit4]   2> Nov 25, 2016 10:06:02 PM WhitespaceTokenizer initialize
   [junit4]   2> INFO: "Whitespace tokenizer successfully initialized"
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=UIMAUpdateRequestProcessorTest -Dtests.method=testErrorHandling -Dtests.seed=1E6F7E32F1994784 -Dtests.slow=true -Dtests.locale=ms -Dtests.timezone=Antarctica/McMurdo -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   5.14s J0 | UIMAUpdateRequestProcessorTest.testErrorHandling <<<
   [junit4]    > Throwable #1: org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
   [junit4]    >   Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([1E6F7E32F1994784:2B9B000CA74FC650]:0)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
   [junit4]    > 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testErrorHandling(UIMAUpdateRequestProcessorTest.java:137)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=UIMAUpdateRequestProcessorTest -Dtests.method=testTwoUpdates -Dtests.seed=1E6F7E32F1994784 -Dtests.slow=true -Dtests.locale=ms -Dtests.timezone=Antarctica/McMurdo -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.01s J0 | UIMAUpdateRequestProcessorTest.testTwoUpdates <<<
   [junit4]    > Throwable #1: org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">1</field><field name="text">The Apache Software Foundation is happy to announce BarCampApache Sydney, Australia, the first ASF-backed event in the Southern Hemisphere!</field></doc></add>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([1E6F7E32F1994784:1325296DE9B6252A]:0)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
   [junit4]    > 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testTwoUpdates(UIMAUpdateRequestProcessorTest.java:101)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Nov 25, 2016 10:06:06 PM WhitespaceTokenizer initialize
   [junit4]   2> INFO: "Whitespace tokenizer successfully initialized"
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=UIMAUpdateRequestProcessorTest -Dtests.method=testProcessing -Dtests.seed=1E6F7E32F1994784 -Dtests.slow=true -Dtests.locale=ms -Dtests.timezone=Antarctica/McMurdo -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.01s J0 | UIMAUpdateRequestProcessorTest.testProcessing <<<
   [junit4]    > Throwable #1: org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
   [junit4]    >   Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([1E6F7E32F1994784:DE17A40B66AC62E]:0)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
   [junit4]    > 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing(UIMAUpdateRequestProcessorTest.java:83)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=UIMAUpdateRequestProcessorTest -Dtests.method=testMultiplierProcessing -Dtests.seed=1E6F7E32F1994784 -Dtests.slow=true -Dtests.locale=ms -Dtests.timezone=Antarctica/McMurdo -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.01s J0 | UIMAUpdateRequestProcessorTest.testMultiplierProcessing <<<
   [junit4]    > Throwable #1: org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
   [junit4]    >   Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([1E6F7E32F1994784:256EB49FF6CD6B70]:0)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
   [junit4]    > 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing(UIMAUpdateRequestProcessorTest.java:83)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testMultiplierProcessing(UIMAUpdateRequestProcessorTest.java:177)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/contrib/solr-uima/test/J0/temp/solr.uima.processor.UIMAUpdateRequestProcessorTest_1E6F7E32F1994784-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=375, maxMBSortInHeap=5.766022074932533, sim=RandomSimilarity(queryNorm=false): {}, locale=ms, timezone=Antarctica/McMurdo
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=3,threads=1,free=139854512,total=485642240
   [junit4]   2> NOTE: All tests run in this JVM: [UIMAUpdateRequestProcessorTest]
   [junit4] Completed [2/2 (1!)] on J0 in 13.42s, 6 tests, 4 errors <<< FAILURES!

[...truncated 51250 lines...]


[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 977 - Still unstable!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/977/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsRecoveryZkTest

Error Message:
ObjectTracker found 1 object(s) that were not released!!! [HdfsTransactionLog] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)  at org.apache.solr.update.HdfsTransactionLog.<init>(HdfsTransactionLog.java:130)  at org.apache.solr.update.HdfsUpdateLog.init(HdfsUpdateLog.java:202)  at org.apache.solr.update.UpdateHandler.<init>(UpdateHandler.java:137)  at org.apache.solr.update.UpdateHandler.<init>(UpdateHandler.java:94)  at org.apache.solr.update.DirectUpdateHandler2.<init>(DirectUpdateHandler2.java:102)  at sun.reflect.GeneratedConstructorAccessor146.newInstance(Unknown Source)  at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)  at java.lang.reflect.Constructor.newInstance(Constructor.java:423)  at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:723)  at org.apache.solr.core.SolrCore.createUpdateHandler(SolrCore.java:785)  at org.apache.solr.core.SolrCore.initUpdateHandler(SolrCore.java:1024)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:889)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:793)  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:868)  at org.apache.solr.core.CoreContainer.lambda$load$0(CoreContainer.java:517)  at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)  at java.lang.Thread.run(Thread.java:745)  

Stack Trace:
java.lang.AssertionError: ObjectTracker found 1 object(s) that were not released!!! [HdfsTransactionLog]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
	at org.apache.solr.update.HdfsTransactionLog.<init>(HdfsTransactionLog.java:130)
	at org.apache.solr.update.HdfsUpdateLog.init(HdfsUpdateLog.java:202)
	at org.apache.solr.update.UpdateHandler.<init>(UpdateHandler.java:137)
	at org.apache.solr.update.UpdateHandler.<init>(UpdateHandler.java:94)
	at org.apache.solr.update.DirectUpdateHandler2.<init>(DirectUpdateHandler2.java:102)
	at sun.reflect.GeneratedConstructorAccessor146.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:723)
	at org.apache.solr.core.SolrCore.createUpdateHandler(SolrCore.java:785)
	at org.apache.solr.core.SolrCore.initUpdateHandler(SolrCore.java:1024)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:889)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:793)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:868)
	at org.apache.solr.core.CoreContainer.lambda$load$0(CoreContainer.java:517)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)


	at __randomizedtesting.SeedInfo.seed([82E6C06A2B1C9E3B]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:266)
	at sun.reflect.GeneratedMethodAccessor30.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 12322 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsRecoveryZkTest
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_82E6C06A2B1C9E3B-001/init-core-data-001
   [junit4]   2> 2419406 INFO  (SUITE-HdfsRecoveryZkTest-seed#[82E6C06A2B1C9E3B]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 2419406 INFO  (SUITE-HdfsRecoveryZkTest-seed#[82E6C06A2B1C9E3B]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_82E6C06A2B1C9E3B-001/tempDir-001
   [junit4]   2> 2419407 INFO  (SUITE-HdfsRecoveryZkTest-seed#[82E6C06A2B1C9E3B]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2419407 INFO  (Thread-7143) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2419407 INFO  (Thread-7143) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2419507 INFO  (SUITE-HdfsRecoveryZkTest-seed#[82E6C06A2B1C9E3B]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:49543
   [junit4]   2> 2419518 INFO  (jetty-launcher-3452-thread-1) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2419519 INFO  (jetty-launcher-3452-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@187fec17{/solr,null,AVAILABLE}
   [junit4]   2> 2419524 INFO  (jetty-launcher-3452-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@3bc7baf9{HTTP/1.1,[http/1.1]}{127.0.0.1:47816}
   [junit4]   2> 2419524 INFO  (jetty-launcher-3452-thread-2) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2419524 INFO  (jetty-launcher-3452-thread-1) [    ] o.e.j.s.Server Started @2423002ms
   [junit4]   2> 2419525 INFO  (jetty-launcher-3452-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=47816}
   [junit4]   2> 2419525 ERROR (jetty-launcher-3452-thread-1) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2419525 INFO  (jetty-launcher-3452-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 2419525 INFO  (jetty-launcher-3452-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2419525 INFO  (jetty-launcher-3452-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2419526 INFO  (jetty-launcher-3452-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2016-11-28T21:42:11.330Z
   [junit4]   2> 2419526 INFO  (jetty-launcher-3452-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@25bf22c0{/solr,null,AVAILABLE}
   [junit4]   2> 2419526 INFO  (jetty-launcher-3452-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4d2b11a7{HTTP/1.1,[http/1.1]}{127.0.0.1:52239}
   [junit4]   2> 2419526 INFO  (jetty-launcher-3452-thread-2) [    ] o.e.j.s.Server Started @2423004ms
   [junit4]   2> 2419526 INFO  (jetty-launcher-3452-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=52239}
   [junit4]   2> 2419526 ERROR (jetty-launcher-3452-thread-2) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2419526 INFO  (jetty-launcher-3452-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 2419526 INFO  (jetty-launcher-3452-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2419526 INFO  (jetty-launcher-3452-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2419527 INFO  (jetty-launcher-3452-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2016-11-28T21:42:11.331Z
   [junit4]   2> 2419533 INFO  (jetty-launcher-3452-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2419533 INFO  (jetty-launcher-3452-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2419542 INFO  (jetty-launcher-3452-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49543/solr
   [junit4]   2> 2419543 INFO  (jetty-launcher-3452-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49543/solr
   [junit4]   2> 2419586 INFO  (jetty-launcher-3452-thread-1) [n:127.0.0.1:47816_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:47816_solr
   [junit4]   2> 2419588 INFO  (jetty-launcher-3452-thread-2) [n:127.0.0.1:52239_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52239_solr
   [junit4]   2> 2419589 INFO  (jetty-launcher-3452-thread-1) [n:127.0.0.1:47816_solr    ] o.a.s.c.Overseer Overseer (id=97017484490440709-127.0.0.1:47816_solr-n_0000000000) starting
   [junit4]   2> 2419592 INFO  (zkCallback-3463-thread-1-processing-n:127.0.0.1:52239_solr) [n:127.0.0.1:52239_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2419593 INFO  (zkCallback-3462-thread-1-processing-n:127.0.0.1:47816_solr) [n:127.0.0.1:47816_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2419610 INFO  (jetty-launcher-3452-thread-1) [n:127.0.0.1:47816_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47816_solr
   [junit4]   2> 2419613 INFO  (zkCallback-3463-thread-1-processing-n:127.0.0.1:52239_solr) [n:127.0.0.1:52239_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2419622 INFO  (zkCallback-3462-thread-1-processing-n:127.0.0.1:47816_solr) [n:127.0.0.1:47816_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2419690 INFO  (jetty-launcher-3452-thread-1) [n:127.0.0.1:47816_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_82E6C06A2B1C9E3B-001/tempDir-001/node1/.
   [junit4]   2> 2419704 INFO  (jetty-launcher-3452-thread-2) [n:127.0.0.1:52239_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_82E6C06A2B1C9E3B-001/tempDir-001/node2/.
   [junit4]   2> 2419715 INFO  (SUITE-HdfsRecoveryZkTest-seed#[82E6C06A2B1C9E3B]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2419717 INFO  (SUITE-HdfsRecoveryZkTest-seed#[82E6C06A2B1C9E3B]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:49543/solr ready
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 2419783 WARN  (SUITE-HdfsRecoveryZkTest-seed#[82E6C06A2B1C9E3B]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 2419795 WARN  (SUITE-HdfsRecoveryZkTest-seed#[82E6C06A2B1C9E3B]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 2419798 INFO  (SUITE-HdfsRecoveryZkTest-seed#[82E6C06A2B1C9E3B]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 2419813 INFO  (SUITE-HdfsRecoveryZkTest-seed#[82E6C06A2B1C9E3B]-worker) [    ] o.m.log Extract jar:file:/export/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs to ./temp/Jetty_solaris.vm_33349_hdfs____.xstal2/webapp
   [junit4]   2> 2420413 INFO  (SUITE-HdfsRecoveryZkTest-seed#[82E6C06A2B1C9E3B]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@solaris-vm:33349
   [junit4]   2> 2420940 WARN  (SUITE-HdfsRecoveryZkTest-seed#[82E6C06A2B1C9E3B]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 2420942 INFO  (SUITE-HdfsRecoveryZkTest-seed#[82E6C06A2B1C9E3B]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 2420964 INFO  (SUITE-HdfsRecoveryZkTest-seed#[82E6C06A2B1C9E3B]-worker) [    ] o.m.log Extract jar:file:/export/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_42015_datanode____ynsvel/webapp
   [junit4]   2> 2421637 INFO  (SUITE-HdfsRecoveryZkTest-seed#[82E6C06A2B1C9E3B]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42015
   [junit4]   2> 2421882 WARN  (SUITE-HdfsRecoveryZkTest-seed#[82E6C06A2B1C9E3B]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 2421889 INFO  (SUITE-HdfsRecoveryZkTest-seed#[82E6C06A2B1C9E3B]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 2421951 INFO  (SUITE-HdfsRecoveryZkTest-seed#[82E6C06A2B1C9E3B]-worker) [    ] o.m.log Extract jar:file:/export/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_41642_datanode____.ux629u/webapp
   [junit4]   2> 2422023 INFO  (IPC Server handler 5 on 52856) [    ] BlockStateChange BLOCK* processReport: from storage DS-705c1dbc-607f-42a8-9204-da5cc54dd080 node DatanodeRegistration(127.0.0.1:47555, datanodeUuid=aea6d05b-a3bf-4d0b-9aa5-3e1844d45213, infoPort=39576, infoSecurePort=0, ipcPort=44818, storageInfo=lv=-56;cid=testClusterID;nsid=802589365;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
   [junit4]   2> 2422023 INFO  (IPC Server handler 5 on 52856) [    ] BlockStateChange BLOCK* processReport: from storage DS-9d395f1c-bd42-4f7b-bd76-2ff55bda6318 node DatanodeRegistration(127.0.0.1:47555, datanodeUuid=aea6d05b-a3bf-4d0b-9aa5-3e1844d45213, infoPort=39576, infoSecurePort=0, ipcPort=44818, storageInfo=lv=-56;cid=testClusterID;nsid=802589365;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 2422599 INFO  (SUITE-HdfsRecoveryZkTest-seed#[82E6C06A2B1C9E3B]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41642
   [junit4]   2> 2422797 INFO  (IPC Server handler 6 on 52856) [    ] BlockStateChange BLOCK* processReport: from storage DS-e379a6ce-a3e2-4096-86b0-fc8b208b8a11 node DatanodeRegistration(127.0.0.1:59595, datanodeUuid=da6714fc-d328-4a09-8cef-c141af1557c2, infoPort=58833, infoSecurePort=0, ipcPort=33983, storageInfo=lv=-56;cid=testClusterID;nsid=802589365;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
   [junit4]   2> 2422797 INFO  (IPC Server handler 6 on 52856) [    ] BlockStateChange BLOCK* processReport: from storage DS-5f719703-649e-4d92-ab60-0390cb946749 node DatanodeRegistration(127.0.0.1:59595, datanodeUuid=da6714fc-d328-4a09-8cef-c141af1557c2, infoPort=58833, infoSecurePort=0, ipcPort=33983, storageInfo=lv=-56;cid=testClusterID;nsid=802589365;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 2423036 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[82E6C06A2B1C9E3B]) [    ] o.a.s.SolrTestCaseJ4 ###Starting test
   [junit4]   2> 2423041 INFO  (qtp1918846368-23384) [n:127.0.0.1:52239_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf&maxShardsPerNode=1&name=recoverytest&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2423045 INFO  (OverseerThreadFactory-9815-thread-1-processing-n:127.0.0.1:47816_solr) [n:127.0.0.1:47816_solr    ] o.a.s.c.CreateCollectionCmd Create collection recoverytest
   [junit4]   2> 2423155 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=recoverytest_shard1_replica1&action=CREATE&numShards=1&collection=recoverytest&shard=shard1&wt=javabin&version=2
   [junit4]   2> 2423155 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=recoverytest_shard1_replica2&action=CREATE&numShards=1&collection=recoverytest&shard=shard1&wt=javabin&version=2
   [junit4]   2> 2423269 INFO  (zkCallback-3463-thread-1-processing-n:127.0.0.1:52239_solr) [n:127.0.0.1:52239_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/recoverytest/state.json] for collection [recoverytest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2423269 INFO  (zkCallback-3462-thread-2-processing-n:127.0.0.1:47816_solr) [n:127.0.0.1:47816_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/recoverytest/state.json] for collection [recoverytest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2424172 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1  x:recoverytest_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2424199 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1  x:recoverytest_shard1_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2424208 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1  x:recoverytest_shard1_replica1] o.a.s.s.IndexSchema [recoverytest_shard1_replica1] Schema name=minimal
   [junit4]   2> 2424211 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1  x:recoverytest_shard1_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2424216 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1  x:recoverytest_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'recoverytest_shard1_replica1' using configuration from collection recoverytest
   [junit4]   2> 2424223 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1  x:recoverytest_shard1_replica2] o.a.s.s.IndexSchema [recoverytest_shard1_replica2] Schema name=minimal
   [junit4]   2> 2424226 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1  x:recoverytest_shard1_replica2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2424229 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1  x:recoverytest_shard1_replica2] o.a.s.c.CoreContainer Creating SolrCore 'recoverytest_shard1_replica2' using configuration from collection recoverytest
   [junit4]   2> 2424235 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://solaris-vm:52856/data
   [junit4]   2> 2424235 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 2424236 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.SolrCore [[recoverytest_shard1_replica1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_82E6C06A2B1C9E3B-001/tempDir-001/node1/recoverytest_shard1_replica1], dataDir=[hdfs://solaris-vm:52856/data/recoverytest/core_node1/data/]
   [junit4]   2> 2424236 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://solaris-vm:52856/data/recoverytest/core_node1/data/snapshot_metadata
   [junit4]   2> 2424270 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://solaris-vm:52856/data
   [junit4]   2> 2424271 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 2424271 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.SolrCore [[recoverytest_shard1_replica2] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_82E6C06A2B1C9E3B-001/tempDir-001/node2/recoverytest_shard1_replica2], dataDir=[hdfs://solaris-vm:52856/data/recoverytest/core_node2/data/]
   [junit4]   2> 2424271 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://solaris-vm:52856/data/recoverytest/core_node2/data/snapshot_metadata
   [junit4]   2> 2424275 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 2424276 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
   [junit4]   2> 2424276 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 2424284 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 2424284 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
   [junit4]   2> 2424284 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 2424293 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 2424297 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 2424299 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://solaris-vm:52856/data/recoverytest/core_node2/data
   [junit4]   2> 2424300 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://solaris-vm:52856/data/recoverytest/core_node1/data
   [junit4]   2> 2424335 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://solaris-vm:52856/data/recoverytest/core_node1/data/index
   [junit4]   2> 2424337 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://solaris-vm:52856/data/recoverytest/core_node2/data/index
   [junit4]   2> 2424346 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 2424346 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
   [junit4]   2> 2424346 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 2424348 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 2424348 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
   [junit4]   2> 2424348 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 2424354 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 2424361 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 2424400 INFO  (IPC Server handler 1 on 52856) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:47555 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-e379a6ce-a3e2-4096-86b0-fc8b208b8a11:NORMAL:127.0.0.1:59595|RBW], ReplicaUC[[DISK]DS-9d395f1c-bd42-4f7b-bd76-2ff55bda6318:NORMAL:127.0.0.1:47555|RBW]]} size 0
   [junit4]   2> 2424405 INFO  (IPC Server handler 0 on 52856) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:59595 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-9d395f1c-bd42-4f7b-bd76-2ff55bda6318:NORMAL:127.0.0.1:47555|RBW], ReplicaUC[[DISK]DS-5f719703-649e-4d92-ab60-0390cb946749:NORMAL:127.0.0.1:59595|FINALIZED]]} size 0
   [junit4]   2> 2424413 INFO  (IPC Server handler 7 on 52856) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:47555 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-e379a6ce-a3e2-4096-86b0-fc8b208b8a11:NORMAL:127.0.0.1:59595|RBW], ReplicaUC[[DISK]DS-705c1dbc-607f-42a8-9204-da5cc54dd080:NORMAL:127.0.0.1:47555|RBW]]} size 0
   [junit4]   2> 2424415 INFO  (IPC Server handler 4 on 52856) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:59595 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-e379a6ce-a3e2-4096-86b0-fc8b208b8a11:NORMAL:127.0.0.1:59595|RBW], ReplicaUC[[DISK]DS-705c1dbc-607f-42a8-9204-da5cc54dd080:NORMAL:127.0.0.1:47555|RBW]]} size 0
   [junit4]   2> 2424437 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 2424438 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2424439 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
   [junit4]   2> 2424456 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 2424456 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2424456 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
   [junit4]   2> 2424461 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2424461 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2424475 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2424475 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2424496 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@23da3e24[recoverytest_shard1_replica2] main]
   [junit4]   2> 2424501 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 2424503 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 2424505 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2424506 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1552279757190594560
   [junit4]   2> 2424519 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
   [junit4]   2> 2424520 INFO  (searcherExecutor-9821-thread-1-processing-n:127.0.0.1:52239_solr x:recoverytest_shard1_replica2 s:shard1 c:recoverytest r:core_node2) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.SolrCore [recoverytest_shard1_replica2] Registered new searcher Searcher@23da3e24[recoverytest_shard1_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2424528 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4efd00a[recoverytest_shard1_replica1] main]
   [junit4]   2> 2424531 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 2424532 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 2424532 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2424532 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1552279757217857536
   [junit4]   2> 2424537 INFO  (searcherExecutor-9820-thread-1-processing-n:127.0.0.1:47816_solr x:recoverytest_shard1_replica1 s:shard1 c:recoverytest r:core_node1) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.SolrCore [recoverytest_shard1_replica1] Registered new searcher Searcher@4efd00a[recoverytest_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2424632 INFO  (zkCallback-3463-thread-1-processing-n:127.0.0.1:52239_solr) [n:127.0.0.1:52239_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/recoverytest/state.json] for collection [recoverytest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2424632 INFO  (zkCallback-3462-thread-3-processing-n:127.0.0.1:47816_solr) [n:127.0.0.1:47816_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/recoverytest/state.json] for collection [recoverytest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2425021 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2425022 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2425022 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/
   [junit4]   2> 2425026 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.PeerSync PeerSync: core=recoverytest_shard1_replica2 url=http://127.0.0.1:52239/solr START replicas=[http://127.0.0.1:47816/solr/recoverytest_shard1_replica1/] nUpdates=100
   [junit4]   2> 2425026 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.PeerSync PeerSync: core=recoverytest_shard1_replica2 url=http://127.0.0.1:52239/solr DONE.  We have no versions.  sync failed.
   [junit4]   2> 2425049 INFO  (qtp778312156-23364) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.S.Request [recoverytest_shard1_replica1]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 2425049 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 2425049 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 2425068 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/ shard1
   [junit4]   2> 2425174 INFO  (zkCallback-3462-thread-2-processing-n:127.0.0.1:47816_solr) [n:127.0.0.1:47816_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/recoverytest/state.json] for collection [recoverytest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2425174 INFO  (zkCallback-3463-thread-1-processing-n:127.0.0.1:52239_solr) [n:127.0.0.1:52239_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/recoverytest/state.json] for collection [recoverytest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2425223 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2425228 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=recoverytest_shard1_replica2&action=CREATE&numShards=1&collection=recoverytest&shard=shard1&wt=javabin&version=2} status=0 QTime=2073
   [junit4]   2> 2425331 INFO  (zkCallback-3463-thread-1-processing-n:127.0.0.1:52239_solr) [n:127.0.0.1:52239_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/recoverytest/state.json] for collection [recoverytest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2425331 INFO  (zkCallback-3462-thread-3-processing-n:127.0.0.1:47816_solr) [n:127.0.0.1:47816_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/recoverytest/state.json] for collection [recoverytest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2425553 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=recoverytest_shard1_replica1&action=CREATE&numShards=1&collection=recoverytest&shard=shard1&wt=javabin&version=2} status=0 QTime=2399
   [junit4]   2> 2425560 INFO  (qtp1918846368-23384) [n:127.0.0.1:52239_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 2425655 INFO  (zkCallback-3463-thread-1-processing-n:127.0.0.1:52239_solr) [n:127.0.0.1:52239_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/recoverytest/state.json] for collection [recoverytest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2425655 INFO  (zkCallback-3462-thread-3-processing-n:127.0.0.1:47816_solr) [n:127.0.0.1:47816_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/recoverytest/state.json] for collection [recoverytest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2426561 INFO  (qtp1918846368-23384) [n:127.0.0.1:52239_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=2&collection.configName=conf&maxShardsPerNode=1&name=recoverytest&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=3521
   [junit4]   2> 2426564 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[82E6C06A2B1C9E3B]) [    ] o.a.s.c.RecoveryZkTest Indexing 1350 documents
   [junit4]   2> 2426578 DEBUG (qtp1918846368-23373) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.HdfsTransactionLog Opening new tlog hdfs tlog{file=hdfs://solaris-vm:52856/data/recoverytest/core_node2/data/tlog/tlog.0000000000000000000 refcount=1}
   [junit4]   2> 2426590 DEBUG (qtp778312156-23362) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.HdfsTransactionLog Opening new tlog hdfs tlog{file=hdfs://solaris-vm:52856/data/recoverytest/core_node1/data/tlog/tlog.0000000000000000000 refcount=1}
   [junit4]   2> 2426613 INFO  (qtp778312156-23362) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[1-0 (1552279759351709696)]} 0 33
   [junit4]   2> 2426614 INFO  (qtp1918846368-23373) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-0 (1552279759351709696)]} 0 47
   [junit4]   2> 2426618 INFO  (qtp778312156-23363) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[1-1 (1552279759402041344)]} 0 3
   [junit4]   2> 2426619 INFO  (qtp1918846368-23374) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-1 (1552279759402041344)]} 0 4
   [junit4]   2> 2426619 INFO  (qtp778312156-23366) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[2-0 (1552279759354855424)]} 0 38
   [junit4]   2> 2426619 INFO  (qtp1918846368-23377) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-0 (1552279759354855424)]} 0 49
   [junit4]   2> 2426623 INFO  (qtp778312156-23361) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{delete=[1-1 (-1552279759408332800)]} 0 1
   [junit4]   2> 2426623 INFO  (qtp1918846368-23385) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-1 (-1552279759408332800)]} 0 3
   [junit4]   2> 2426624 INFO  (qtp778312156-23367) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[2-1 (1552279759408332801)]} 0 1
   [junit4]   2> 2426624 INFO  (qtp1918846368-23376) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-1 (1552279759408332801)]} 0 2
   [junit4]   2> 2426627 INFO  (qtp778312156-23364) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[1-2 (1552279759411478528)]} 0 1
   [junit4]   2> 2426627 INFO  (qtp1918846368-23383) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-2 (1552279759411478528)]} 0 3
   [junit4]   2> 2426628 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{delete=[2-1 (-1552279759412527104)]} 0 1
   [junit4]   2> 2426628 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[2-1 (-1552279759412527104)]} 0 3
   [junit4]   2> 2426631 INFO  (qtp778312156-23363) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[2-2 (1552279759416721409)]} 0 1
   [junit4]   2> 2426631 INFO  (qtp778312156-23362) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[1-3 (1552279759416721408)]} 0 1
   [junit4]   2> 2426632 INFO  (qtp1918846368-23384) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-3 (1552279759416721408)]} 0 3
   [junit4]   2> 2426632 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-2 (1552279759416721409)]} 0 2
   [junit4]   2> 2426642 INFO  (qtp778312156-23361) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[2-3 (1552279759424061440)]} 0 5
   [junit4]   2> 2426642 INFO  (qtp778312156-23366) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{delete=[1-3 (-1552279759420915712)]} 0 9
   [junit4]   2> 2426643 INFO  (qtp1918846368-23374) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-3 (1552279759424061440)]} 0 7
   [junit4]   2> 2426643 INFO  (qtp1918846368-23373) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-3 (-1552279759420915712)]} 0 10
   [junit4]   2> 2426646 INFO  (qtp778312156-23367) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{delete=[2-3 (-1552279759432450048)]} 0 1
   [junit4]   2> 2426647 INFO  (qtp1918846368-23377) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[2-3 (-1552279759432450048)]} 0 3
   [junit4]   2> 2426651 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[1-4 (1552279759436644352)]} 0 1
   [junit4]   2> 2426652 INFO  (qtp778312156-23364) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[2-4 (1552279759436644353)]} 0 2
   [junit4]   2> 2426652 INFO  (qtp1918846368-23377) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-4 (1552279759436644352)]} 0 4
   [junit4]   2> 2426652 INFO  (qtp1918846368-23385) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-4 (1552279759436644353)]} 0 4
   [junit4]   2> 2426656 INFO  (qtp778312156-23363) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{delete=[2-4 (-1552279759441887232)]} 0 1
   [junit4]   2> 2426657 INFO  (qtp1918846368-23376) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[2-4 (-1552279759441887232)]} 0 3
   [junit4]   2> 2426660 INFO  (qtp778312156-23363) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{delete=[1-4 (-1552279759443984384)]} 0 3
   [junit4]   2> 2426660 INFO  (qtp1918846368-23383) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-4 (-1552279759443984384)]} 0 5
   [junit4]   2> 2426665 INFO  (qtp778312156-23362) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[2-5 (1552279759447130112)]} 0 7
   [junit4]   2> 2426666 INFO  (qtp1918846368-23371) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-5 (1552279759447130112)]} 0 8
   [junit4]   2> 2426667 INFO  (qtp778312156-23361) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[1-5 (1552279759454470144)]} 0 1
   [junit4]   2> 2426668 INFO  (qtp1918846368-23384) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-5 (1552279759454470144)]} 0 3
   [junit4]   2> 2426669 INFO  (qtp778312156-23362) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{delete=[2-5 (-1552279759456567296)]} 0 1
   [junit4]   2> 2426669 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[2-5 (-1552279759456567296)]} 0 2
   [junit4]   2> 2426673 INFO  (qtp778312156-23361) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{delete=[1-5 (-1552279759458664448)]} 0 3
   [junit4]   2> 2426674 INFO  (qtp1918846368-23374) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-5 (-1552279759458664448)]} 0 5
   [junit4]   2> 2426675 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[2-6 (1552279759461810176)]} 0 1
   [junit4]   2> 2426675 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-6 (1552279759461810176)]} 0 3
   [junit4]   2> 2426677 INFO  (qtp778312156-23364) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[1-6 (1552279759464955904)]} 0 1
   [junit4]   2> 2426678 INFO  (qtp1918846368-23377) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-6 (1552279759464955904)]} 0 3
   [junit4]   2> 2426680 INFO  (qtp778312156-23363) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{delete=[2-6 (-1552279759467053056)]} 0 1
   [junit4]   2> 2426680 INFO  (qtp1918846368-23385) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[2-6 (-1552279759467053056)]} 0 3
   [junit4]   2> 2426680 INFO  (qtp778312156-23366) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{delete=[1-6 (-1552279759469150208)]} 0 0
   [junit4]   2> 2426681 INFO  (qtp1918846368-23376) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-6 (-1552279759469150208)]} 0 1
   [junit4]   2> 2426683 INFO  (qtp778312156-23367) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[2-7 (1552279759471247360)]} 0 1
   [junit4]   2> 2426683 INFO  (qtp1918846368-23383) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-7 (1552279759471247360)]} 0 2
   [junit4]   2> 2426683 INFO  (qtp778312156-23362) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[1-7 (1552279759471247361)]} 0 1
   [junit4]   2> 2426684 INFO  (qtp1918846368-23376) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-7 (1552279759471247361)]} 0 2
   [junit4]   2> 2426687 INFO  (qtp778312156-23361) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[2-8 (1552279759475441664)]} 0 1
   [junit4]   2> 2426688 INFO  (qtp1918846368-23371) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-8 (1552279759475441664)]} 0 3
   [junit4]   2> 2426689 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[1-8 (1552279759475441665)]} 0 2
   [junit4]   2> 2426690 INFO  (qtp1918846368-23384) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-8 (1552279759475441665)]} 0 4
   [junit4]   2> 2426691 INFO  (qtp778312156-23364) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[2-9 (1552279759479635968)]} 0 1
   [junit4]   2> 2426691 INFO  (qtp1918846368-23373) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-9 (1552279759479635968)]} 0 2
   [junit4]   2> 2426692 INFO  (qtp778312156-23363) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[1-9 (1552279759480684544)]} 0 1
   [junit4]   2> 2426693 INFO  (qtp1918846368-23374) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-9 (1552279759480684544)]} 0 2
   [junit4]   2> 2426694 INFO  (qtp778312156-23366) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{delete=[2-7 (-1552279759482781696)]} 0 1
   [junit4]   2> 2426694 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[2-7 (-1552279759482781696)]} 0 2
   [junit4]   2> 2426696 INFO  (qtp778312156-23367) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{delete=[1-7 (-1552279759484878848)]} 0 1
   [junit4]   2> 2426697 INFO  (qtp1918846368-23377) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-7 (-1552279759484878848)]} 0 3
   [junit4]   2> 2426699 INFO  (qtp778312156-23362) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[2-10 (1552279759485927424)]} 0 2
   [junit4]   2> 2426699 INFO  (qtp1918846368-23385) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-10 (1552279759485927424)]} 0 3
   [junit4]   2> 2426699 INFO  (qtp778312156-23361) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[1-10 (1552279759489073152)]} 0 1
   [junit4]   2> 2426700 INFO  (qtp1918846368-23383) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-10 (1552279759489073152)]} 0 2
   [junit4]   2> 2426705 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{delete=[1-8 (-1552279759492218880)]} 0 2
   [junit4]   2> 2426705 INFO  (qtp1918846368-23376) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-8 (-1552279759492218880)]} 0 4
   [junit4]   2> 2426706 INFO  (qtp778312156-23364) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{delete=[2-8 (-1552279759493267456)]} 0 1
   [junit4]   2> 2426706 INFO  (qtp1918846368-23371) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[2-8 (-1552279759493267456)]} 0 5
   [junit4]   2> 2426708 INFO  (qtp778312156-23363) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[1-11 (1552279759497461760)]} 0 1
   [junit4]   2> 2426709 INFO  (qtp1918846368-23376) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-11 (1552279759497461760)]} 0 2
   [junit4]   2> 2426710 INFO  (qtp778312156-23366) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[2-11 (1552279759498510336)]} 0 2
   [junit4]   2> 2426710 INFO  (qtp1918846368-23384) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-11 (1552279759498510336)]} 0 3
   [junit4]   2> 2426713 INFO  (qtp778312156-23367) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{delete=[1-10 (-1552279759501656064)]} 0 2
   [junit4]   2> 2426713 INFO  (qtp1918846368-23373) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-10 (-1552279759501656064)]} 0 3
   [junit4]   2> 2426714 INFO  (qtp778312156-23362) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{delete=[2-10 (-1552279759502704640)]} 0 2
   [junit4]   2> 2426714 INFO  (qtp1918846368-23374) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[2-10 (-1552279759502704640)]} 0 3
   [junit4]   2> 2426717 INFO  (qtp778312156-23361) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[1-12 (1552279759505850368)]} 0 1
   [junit4]   2> 2426717 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-12 (1552279759505850368)]} 0 3
   [junit4]   2> 2426718 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[2-12 (1552279759506898944)]} 0 2
   [junit4]   2> 2426718 INFO  (qtp1918846368-23374) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-12 (1552279759506898944)]} 0 3
   [junit4]   2> 2426720 INFO  (qtp778312156-23364) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[1-13 (1552279759510044672)]} 0 0
   [junit4]   2> 2426720 INFO  (qtp1918846368-23385) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-13 (1552279759510044672)]} 0 1
   [junit4]   2> 2426722 INFO  (qtp778312156-23363) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[2-13 (1552279759511093248)]} 0 2
   [junit4]   2> 2426725 INFO  (qtp778312156-23366) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[1-14 (1552279759513190400)]} 0 1
   [junit4]   2> 2426726 INFO  (qtp1918846368-23371) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-14 (1552279759513190400)]} 0 4
   [junit4]   2> 2426728 INFO  (qtp778312156-23367) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[1-15 (1552279759519481856)]} 0 1
   [junit4]   2> 2426729 INFO  (qtp1918846368-23376) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-15 (1552279759519481856)]} 0 2
   [junit4]   2> 2426729 INFO  (qtp1918846368-23383) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-13 (1552279759511093248)]} 0 9
   [junit4]   2> 2426734 INFO  (qtp778312156-23361) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[2-14 (1552279759523676160)]} 0 1
   [junit4]   2> 2426734 INFO  (qtp778312156-23362) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{delete=[1-13 (-1552279759522627584)]} 0 3
   [junit4]   2> 2426735 INFO  (qtp1918846368-23384) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-13 (-1552279759522627584)]} 0 4
   [junit4]   2> 2426737 INFO  (qtp1918846368-23373) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-14 (1552279759523676160)]} 0 5
   [junit4]   2> 2426739 INFO  (qtp778312156-23364) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[2-15 (1552279759529967616)]} 0 1
   [junit4]   2> 2426740 INFO  (qtp1918846368-23373) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-15 (1552279759529967616)]} 0 2
   [junit4]   2> 2426740 INFO  (qtp778312156-23365) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[1-16 (1552279759528919040)]} 0 3
   [junit4]   2> 2426740 INFO  (qtp1918846368-23377) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-16 (1552279759528919040)]} 0 4
   [junit4]   2> 2426746 INFO  (qtp778312156-23366) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{delete=[1-14 (-1552279759536259072)]} 0 2
   [junit4]   2> 2426747 INFO  (qtp778312156-23363) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{delete=[2-13 (-1552279759534161920)]} 0 5
   [junit4]   2> 2426747 INFO  (qtp1918846368-23374) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[2-13 (-1552279759534161920)]} 0 6
   [junit4]   2> 2426749 INFO  (qtp1918846368-23377) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-14 (-1552279759536259072)]} 0 5
   [junit4]   2> 2426751 INFO  (qtp778312156-23361) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[2-16 (1552279759543599105)]} 0 0
   [junit4]   2> 2426752 INFO  (qtp778312156-23367) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[1-17 (1552279759543599104)]} 0 1
   [junit4]   2> 2426752 INFO  (qtp1918846368-23383) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-16 (1552279759543599105)]} 0 1
   [junit4]   2> 2426752 INFO  (qtp1918846368-23371) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-17 (1552279759543599104)]} 0 2
   [junit4]   2> 2426755 INFO  (qtp778312156-23364) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{delete=[2-14 (-1552279759547793408)]} 0 0
   [junit4]   2> 2426756 INFO  (qtp778312156-23362) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[1-18 (1552279759546744832)]} 0 2
   [junit4]   2> 2426756 INFO  (qtp1918846368-23384) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[2-14 (-1552279759547793408)]} 0 2
   [junit4]   2> 2426756 INFO  (qtp1918846368-23376) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-18 (1552279759546744832)]} 0 3
   [junit4]   2> 2426761 INFO  (qtp778312156-23364) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{delete=[1-16 (-1552279759551987712)]} 0 1
   [junit4]   2> 2426761 INFO  (qtp778312156-23366) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[2-17 (1552279759553036288)]} 0 1
   [junit4]   2> 2426761 INFO  (qtp1918846368-23373) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-17 (1552279759553036288)]} 0 2
   [junit4]   2> 2426762 INFO  (qtp1918846368-23375) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-16 (-1552279759551987712)]} 0 3
   [junit4]   2> 2426764 INFO  (qtp778312156-23363) [n:127.0.0.1:47816_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52239/solr/recoverytest_shard1_replica2/&wt=javabin&version=2}{add=[1-19 (1552279759556182016)]} 0 1
   [junit4]   2> 2426766 ERROR (updateExecutor-3456-thread-3-processing-http:////127.0.0.1:47816//solr//recoverytest_shard1_replica1 x:recoverytest_shard1_replica2 r:core_node2 n:127.0.0.1:52239_solr s:shard1 c:recoverytest) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.StreamingSolrClients error
   [junit4]   2> org.apache.http.NoHttpResponseException: 127.0.0.1:47816 failed to respond
   [junit4]   2> 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143)
   [junit4]   2> 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
   [junit4]   2> 	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
   [junit4]   2> 	at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165)
   [junit4]   2> 	at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:167)
   [junit4]   2> 	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
   [junit4]   2> 	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
   [junit4]   2> 	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:271)
   [junit4]   2> 	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
   [junit4]   2> 	at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
   [junit4]   2> 	at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
   [junit4]   2> 	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
   [junit4]   2> 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   [junit4]   2> 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]   2> 	at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.sendUpdateStream(ConcurrentUpdateSolrClient.java:324)
   [junit4]   2> 	at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.run(ConcurrentUpdateSolrClient.java:187)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2426768 WARN  (qtp1918846368-23374) [n:127.0.0.1:52239_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.DistributedUpdateProcessor Error sending update to http://127.0.0.1:47816/solr
   [junit4]   2> org.apache.http.NoHttpResponseException: 127.0.0.1:47816 failed to respond
   [junit4]   2> 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143)
   [junit4]   2> 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
   [junit4]   2> 	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
   [junit4]   2> 	at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165)
   [junit4]   2> 	at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:167)
   [junit4]   2> 	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
   [junit4]   2> 	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
   [junit4]   2> 	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:271)
   [junit4]   2> 	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
   [junit4]   2> 	at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
   [junit4]   2> 	at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
   [junit4]   2> 	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
   [junit4]   2> 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   [junit4]   2> 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]   2> 	at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.sendUpdateStream(ConcurrentUpdateSolrClient.java:324)
   [junit4]   2> 	at org.apache.solr

[...truncated too long message...]

randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:203)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3370)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:5729)
   [junit4]   2> 	... 54 more
   [junit4]   2> 2445606 INFO  (SUITE-HdfsRecoveryZkTest-seed#[82E6C06A2B1C9E3B]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:49543 49543
   [junit4]   2> 2445606 INFO  (SUITE-HdfsRecoveryZkTest-seed#[82E6C06A2B1C9E3B]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {rnd_b=PostingsFormat(name=Memory), _version_=Lucene50(blocksize=128), a_t=PostingsFormat(name=LuceneVarGapFixedInterval), a_i=PostingsFormat(name=Memory), id=PostingsFormat(name=Memory)}, docValues:{}, maxPointsInLeafNode=1760, maxMBSortInHeap=6.4468187740192935, sim=RandomSimilarity(queryNorm=true): {}, locale=th, timezone=Asia/Bishkek
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=3,threads=2,free=183910872,total=536870912
   [junit4]   2> NOTE: All tests run in this JVM: [TestHdfsBackupRestoreCore, TestAtomicUpdateErrorCases, CollectionsAPISolrJTest, TestIndexingPerformance, TlogReplayBufferedWhileIndexingTest, TestSolrFieldCacheMBean, TestDownShardTolerantSearch, BlockJoinFacetDistribTest, TestCrossCoreJoin, SortByFunctionTest, TestNumericTerms32, TestReload, TestManagedResourceStorage, TestFuzzyAnalyzedSuggestions, TestSchemaManager, DistributedExpandComponentTest, BufferStoreTest, FullHLLTest, SmileWriterTest, TestSimpleTrackingShardHandler, TestAuthorizationFramework, PreAnalyzedFieldManagedSchemaCloudTest, CSVRequestHandlerTest, TestDFISimilarityFactory, AddSchemaFieldsUpdateProcessorFactoryTest, ClassificationUpdateProcessorFactoryTest, DistributedVersionInfoTest, TestSuggestSpellingConverter, TestSerializedLuceneMatchVersion, DistributedFacetPivotLargeTest, DistribCursorPagingTest, NotRequiredUniqueKeyTest, BasicAuthStandaloneTest, DistanceFunctionTest, TestSubQueryTransformerDistrib, TestQuerySenderListener, TestSolrCoreSnapshots, TestSmileRequest, TestComponentsName, TestValueSourceCache, ResponseLogComponentTest, TestFieldCollectionResource, TestMaxScoreQueryParser, HdfsChaosMonkeySafeLeaderTest, ScriptEngineTest, HttpSolrCallGetCoreTest, HdfsTlogReplayBufferedWhileIndexingTest, ForceLeaderTest, TestJsonRequest, HdfsDirectoryFactoryTest, HdfsLockFactoryTest, AutoCommitTest, ShowFileRequestHandlerTest, TimerUtilsTest, OverriddenZkACLAndCredentialsProvidersTest, DocValuesMissingTest, TestSystemIdResolver, ConjunctionSolrSpellCheckerTest, SpellCheckCollatorTest, TemplateUpdateProcessorTest, TestTestInjection, TestEmbeddedSolrServerConstructors, DefaultValueUpdateProcessorTest, TestReloadAndDeleteDocs, TermVectorComponentTest, TestSolrQueryResponse, DateMathParserTest, MergeStrategyTest, CoreAdminCreateDiscoverTest, TestManagedSchemaAPI, CdcrReplicationHandlerTest, TestJmxMonitoredMap, RAMDirectoryFactoryTest, StatsReloadRaceTest, TestDocBasedVersionConstraints, TestBulkSchemaAPI, TestStressUserVersions, TestTrie, ConfigSetsAPITest, SignatureUpdateProcessorFactoryTest, TestSizeLimitedDistributedMap, HdfsSyncSliceTest, TestLeaderElectionZkExpiry, ConnectionReuseTest, HdfsBasicDistributedZkTest, TestInitQParser, DeleteNodeTest, TestHdfsUpdateLog, CollectionsAPIDistributedZkTest, TestDistributedStatsComponentCardinality, ZkStateWriterTest, TestCustomDocTransformer, FacetPivotSmallTest, TestLRUStatsCache, DataDrivenBlockJoinTest, TestBulkSchemaConcurrent, TestLMDirichletSimilarityFactory, TestRangeQuery, ClassificationUpdateProcessorTest, DirectUpdateHandlerOptimizeTest, BitVectorTest, RemoteQueryErrorTest, SaslZkACLProviderTest, TestMiniSolrCloudCluster, JavabinLoaderTest, StatsComponentTest, UUIDFieldTest, SolrInfoMBeanTest, TestGeoJSONResponseWriter, VMParamsZkACLAndCredentialsProvidersTest, TestDistributedGrouping, DistributedTermsComponentTest, TestCollationFieldDocValues, ResourceLoaderTest, CurrencyFieldOpenExchangeTest, FieldAnalysisRequestHandlerTest, CleanupOldIndexTest, TestPerFieldSimilarityWithDefaultOverride, SuggesterTest, TermVectorComponentDistributedTest, CollectionReloadTest, BlobRepositoryCloudTest, SpellPossibilityIteratorTest, TestSolrCloudWithKerberosAlt, HdfsNNFailoverTest, BadComponentTest, TestBlobHandler, TestGraphTermsQParserPlugin, TestJsonFacetRefinement, IndexSchemaTest, CreateCollectionCleanupTest, DistributedIntervalFacetingTest, TestSQLHandler, TestSolrQueryParser, LeaderElectionIntegrationTest, HLLUtilTest, AnalyticsQueryTest, DirectoryFactoryTest, UpdateRequestProcessorFactoryTest, LoggingHandlerTest, TestLuceneMatchVersion, HdfsWriteToMultipleCollectionsTest, BasicFunctionalityTest, TestFieldCacheSanityChecker, TestCloudPseudoReturnFields, TestLocalFSCloudBackupRestore, TestOmitPositions, LeaderInitiatedRecoveryOnShardRestartTest, TestSortByMinMaxFunction, UpdateParamsTest, TestManagedStopFilterFactory, TestJavabinTupleStreamParser, TestReqParamsAPI, TestDefaultStatsCache, SecurityConfHandlerTest, DistributedFacetPivotWhiteBoxTest, TestIndexSearcher, NoCacheHeaderTest, TestFoldingMultitermQuery, TestDocSet, TestAuthenticationFramework, BasicDistributedZkTest, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, UnloadDistributedZkTest, BasicZkTest, RecoveryZkTest, TestReplicationHandler, ZkCLITest, TestRecovery, TestStressVersions, TestCoreContainer, TestSolr4Spatial, SolrCmdDistributorTest, BadIndexSchemaTest, TestLazyCores, HighlighterTest, AnalysisAfterCoreReloadTest, CoreAdminHandlerTest, TestStressLucene, SuggesterWFSTTest, SolrCoreCheckLockOnStartupTest, TestUpdate, StatelessScriptUpdateProcessorFactoryTest, DebugComponentTest, TestSurroundQueryParser, TestQueryUtils, StandardRequestHandlerTest, TestWriterPerf, TestQueryTypes, PrimitiveFieldTypeTest, TermsComponentTest, RequestHandlersTest, RequiredFieldsTest, FastVectorHighlighterTest, TestJmxIntegration, ReturnFieldsTest, QueryParsingTest, JsonLoaderTest, PingRequestHandlerTest, HighlighterConfigTest, SOLR749Test, TestSolrIndexConfig, TestQuerySenderNoQuery, SolrIndexConfigTest, TestStressRecovery, CdcrRequestHandlerTest, CdcrVersionReplicationTest, CollectionsAPIAsyncDistributedZkTest, DeleteInactiveReplicaTest, DeleteLastCustomShardedReplicaTest, DeleteReplicaTest, DeleteShardTest, DocValuesNotIndexedTest, LeaderFailureAfterFreshStartTest, OverseerModifyCollectionTest, OverseerStatusTest, PeerSyncReplicationTest, SharedFSAutoReplicaFailoverTest, SimpleCollectionCreateDeleteTest, SolrCloudExampleTest, TestCloudInspectUtil, TestCloudRecovery, TestClusterProperties, TestCollectionAPI, TestConfigSetsAPI, TestConfigSetsAPIZkFailure, TestCryptoKeys, TestDeleteCollectionOnDownNodes, TestExclusionRuleCollectionAccess, TestLeaderElectionWithEmptyReplica, TestLockTree, TestMiniSolrCloudClusterSSL, TestRebalanceLeaders, TestStressCloudBlindAtomicUpdates, HdfsBasicDistributedZk2Test, HdfsChaosMonkeyNothingIsSafeTest, HdfsRecoveryZkTest]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=HdfsRecoveryZkTest -Dtests.seed=82E6C06A2B1C9E3B -Dtests.slow=true -Dtests.locale=th -Dtests.timezone=Asia/Bishkek -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.00s J1 | HdfsRecoveryZkTest (suite) <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: ObjectTracker found 1 object(s) that were not released!!! [HdfsTransactionLog]
   [junit4]    > org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
   [junit4]    > 	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
   [junit4]    > 	at org.apache.solr.update.HdfsTransactionLog.<init>(HdfsTransactionLog.java:130)
   [junit4]    > 	at org.apache.solr.update.HdfsUpdateLog.init(HdfsUpdateLog.java:202)
   [junit4]    > 	at org.apache.solr.update.UpdateHandler.<init>(UpdateHandler.java:137)
   [junit4]    > 	at org.apache.solr.update.UpdateHandler.<init>(UpdateHandler.java:94)
   [junit4]    > 	at org.apache.solr.update.DirectUpdateHandler2.<init>(DirectUpdateHandler2.java:102)
   [junit4]    > 	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
   [junit4]    > 	at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:723)
   [junit4]    > 	at org.apache.solr.core.SolrCore.createUpdateHandler(SolrCore.java:785)
   [junit4]    > 	at org.apache.solr.core.SolrCore.initUpdateHandler(SolrCore.java:1024)
   [junit4]    > 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:889)
   [junit4]    > 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:793)
   [junit4]    > 	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:868)
   [junit4]    > 	at org.apache.solr.core.CoreContainer.lambda$load$0(CoreContainer.java:517)
   [junit4]    > 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]    > 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([82E6C06A2B1C9E3B]:0)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:266)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4] Completed [533/658 (1!)] on J1 in 57.36s, 1 test, 1 failure <<< FAILURES!

[...truncated 62743 lines...]


[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 976 - Failure!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/976/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestLazyCores

Error Message:
ObjectTracker found 5 object(s) that were not released!!! [SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:955)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:793)  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:868)  at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1139)  at org.apache.solr.core.TestLazyCores.testCachingLimit(TestLazyCores.java:203)  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)  at java.lang.reflect.Method.invoke(Method.java:498)  at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)  at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)  at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)  at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)  at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)  at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)  at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)  at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)  at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)  at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)  at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)  at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)  at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)  at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)  at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)  at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)  at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)  at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)  at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)  at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)  at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)  at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)  at java.lang.Thread.run(Thread.java:745)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)  at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)  at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:434)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:859)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:793)  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:868)  at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1139)  at org.apache.solr.core.TestLazyCores.testCachingLimit(TestLazyCores.java:203)  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)  at java.lang.reflect.Method.invoke(Method.java:498)  at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)  at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)  at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)  at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)  at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)  at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)  at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)  at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)  at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)  at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)  at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)  at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)  at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)  at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)  at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)  at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)  at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)  at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)  at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)  at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)  at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)  at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)  at java.lang.Thread.run(Thread.java:745)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)  at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)  at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:332)  at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:659)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:865)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:793)  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:868)  at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1139)  at org.apache.solr.core.TestLazyCores.testCachingLimit(TestLazyCores.java:203)  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)  at java.lang.reflect.Method.invoke(Method.java:498)  at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)  at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)  at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)  at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)  at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)  at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)  at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)  at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)  at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)  at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)  at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)  at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)  at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)  at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)  at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)  at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)  at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)  at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)  at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)  at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)  at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)  at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)  at java.lang.Thread.run(Thread.java:745)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)  at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)  at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:66)  at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:691)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:865)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:793)  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:868)  at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1139)  at org.apache.solr.core.TestLazyCores.testCachingLimit(TestLazyCores.java:203)  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)  at java.lang.reflect.Method.invoke(Method.java:498)  at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)  at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)  at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)  at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)  at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)  at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)  at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)  at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)  at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)  at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)  at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)  at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)  at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)  at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)  at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)  at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)  at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)  at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)  at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)  at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)  at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)  at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)  at java.lang.Thread.run(Thread.java:745)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:816)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:793)  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:868)  at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1139)  at org.apache.solr.core.TestLazyCores.testCachingLimit(TestLazyCores.java:203)  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)  at java.lang.reflect.Method.invoke(Method.java:498)  at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)  at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)  at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)  at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)  at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)  at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)  at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)  at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)  at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)  at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)  at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)  at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)  at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)  at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)  at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)  at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)  at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)  at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)  at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)  at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)  at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)  at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)  at java.lang.Thread.run(Thread.java:745)  

Stack Trace:
java.lang.AssertionError: ObjectTracker found 5 object(s) that were not released!!! [SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:955)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:793)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:868)
	at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1139)
	at org.apache.solr.core.TestLazyCores.testCachingLimit(TestLazyCores.java:203)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
	at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:434)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:859)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:793)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:868)
	at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1139)
	at org.apache.solr.core.TestLazyCores.testCachingLimit(TestLazyCores.java:203)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
	at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:332)
	at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:659)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:865)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:793)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:868)
	at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1139)
	at org.apache.solr.core.TestLazyCores.testCachingLimit(TestLazyCores.java:203)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
	at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:66)
	at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:691)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:865)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:793)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:868)
	at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1139)
	at org.apache.solr.core.TestLazyCores.testCachingLimit(TestLazyCores.java:203)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:816)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:793)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:868)
	at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1139)
	at org.apache.solr.core.TestLazyCores.testCachingLimit(TestLazyCores.java:203)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


	at __randomizedtesting.SeedInfo.seed([AFE15B4084CE5BF5]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:266)
	at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestLazyCores

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.core.TestLazyCores:     1) Thread[id=44959, name=searcherExecutor-9258-thread-1, state=WAITING, group=TGRP-TestLazyCores]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.core.TestLazyCores: 
   1) Thread[id=44959, name=searcherExecutor-9258-thread-1, state=WAITING, group=TGRP-TestLazyCores]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([AFE15B4084CE5BF5]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestLazyCores

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=44959, name=searcherExecutor-9258-thread-1, state=WAITING, group=TGRP-TestLazyCores]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=44959, name=searcherExecutor-9258-thread-1, state=WAITING, group=TGRP-TestLazyCores]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([AFE15B4084CE5BF5]:0)


FAILED:  org.apache.solr.update.HardAutoCommitTest.testCommitWithin

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([AFE15B4084CE5BF5:1533343807E0B5E0]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:818)
	at org.apache.solr.update.HardAutoCommitTest.testCommitWithin(HardAutoCommitTest.java:100)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
</response>

	request was:q=id:529&qt=standard&start=0&rows=20&version=2.2
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:811)
	... 40 more




Build Log:
[...truncated 11485 lines...]
   [junit4] Suite: org.apache.solr.update.HardAutoCommitTest
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.update.HardAutoCommitTest_AFE15B4084CE5BF5-001/init-core-data-001
   [junit4]   2> 2099273 INFO  (SUITE-HardAutoCommitTest-seed#[AFE15B4084CE5BF5]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 2099274 INFO  (SUITE-HardAutoCommitTest-seed#[AFE15B4084CE5BF5]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 2099274 INFO  (SUITE-HardAutoCommitTest-seed#[AFE15B4084CE5BF5]-worker) [    ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/lib, /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 2099303 INFO  (SUITE-HardAutoCommitTest-seed#[AFE15B4084CE5BF5]-worker) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2099321 INFO  (SUITE-HardAutoCommitTest-seed#[AFE15B4084CE5BF5]-worker) [    ] o.a.s.s.IndexSchema [null] Schema name=test
   [junit4]   2> 2099385 WARN  (SUITE-HardAutoCommitTest-seed#[AFE15B4084CE5BF5]-worker) [    ] o.a.s.s.IndexSchema [null] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
   [junit4]   2> 2099386 INFO  (SUITE-HardAutoCommitTest-seed#[AFE15B4084CE5BF5]-worker) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 2099398 INFO  (coreLoadExecutor-8586-thread-1) [    ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/lib, /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 2099431 INFO  (coreLoadExecutor-8586-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2099448 INFO  (coreLoadExecutor-8586-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2099509 WARN  (coreLoadExecutor-8586-thread-1) [    ] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
   [junit4]   2> 2099511 INFO  (coreLoadExecutor-8586-thread-1) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 2099517 INFO  (coreLoadExecutor-8586-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1
   [junit4]   2> 2099518 INFO  (coreLoadExecutor-8586-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.update.HardAutoCommitTest_AFE15B4084CE5BF5-001/init-core-data-001/]
   [junit4]   2> 2099518 INFO  (coreLoadExecutor-8586-thread-1) [    x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2b2e58a6
   [junit4]   2> 2099529 INFO  (coreLoadExecutor-8586-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=45, maxMergeAtOnceExplicit=36, maxMergedSegmentMB=57.55078125, floorSegmentMB=0.400390625, forceMergeDeletesPctAllowed=6.815962730712703, segmentsPerTier=22.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.887416540676816
   [junit4]   2> 2099541 INFO  (coreLoadExecutor-8586-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2099542 INFO  (coreLoadExecutor-8586-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2099542 INFO  (coreLoadExecutor-8586-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2099542 INFO  (coreLoadExecutor-8586-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2099543 INFO  (coreLoadExecutor-8586-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=18, maxMergeAtOnceExplicit=11, maxMergedSegmentMB=11.5810546875, floorSegmentMB=0.947265625, forceMergeDeletesPctAllowed=13.976820766279328, segmentsPerTier=24.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4182004591474424
   [junit4]   2> 2099543 INFO  (coreLoadExecutor-8586-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4f706c83[collection1] main]
   [junit4]   2> 2099543 INFO  (coreLoadExecutor-8586-thread-1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 2101303 INFO  (coreLoadExecutor-8586-thread-1) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 2101370 INFO  (coreLoadExecutor-8586-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2101371 INFO  (coreLoadExecutor-8586-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1552236338857639936
   [junit4]   2> 2101373 INFO  (searcherExecutor-8587-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 2101374 INFO  (searcherExecutor-8587-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 2101374 INFO  (searcherExecutor-8587-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 2101374 INFO  (searcherExecutor-8587-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 2101374 INFO  (searcherExecutor-8587-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 2101375 INFO  (searcherExecutor-8587-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 2101375 INFO  (searcherExecutor-8587-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 2101376 INFO  (searcherExecutor-8587-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 2101376 INFO  (searcherExecutor-8587-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 2101377 INFO  (searcherExecutor-8587-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4f706c83[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2101450 INFO  (SUITE-HardAutoCommitTest-seed#[AFE15B4084CE5BF5]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 2101454 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[AFE15B4084CE5BF5]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testCommitWithin
   [junit4]   2> 2101457 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[AFE15B4084CE5BF5]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{deleteByQuery=*:* (-1552236338944671744)} 0 2
   [junit4]   2> 2101459 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[AFE15B4084CE5BF5]) [    ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/lib, /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 2101496 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[AFE15B4084CE5BF5]) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2101525 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[AFE15B4084CE5BF5]) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2101587 WARN  (TEST-HardAutoCommitTest.testCommitWithin-seed#[AFE15B4084CE5BF5]) [    ] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
   [junit4]   2> 2101589 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[AFE15B4084CE5BF5]) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 2101596 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[AFE15B4084CE5BF5]) [    ] o.a.s.c.CoreContainer Reloading SolrCore 'collection1' using configuration from instancedir /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1
   [junit4]   2> 2101597 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[AFE15B4084CE5BF5]) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.update.HardAutoCommitTest_AFE15B4084CE5BF5-001/init-core-data-001/]
   [junit4]   2> 2101597 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[AFE15B4084CE5BF5]) [    x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2b2e58a6
   [junit4]   2> 2101611 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[AFE15B4084CE5BF5]) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2101611 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[AFE15B4084CE5BF5]) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2101659 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[AFE15B4084CE5BF5]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3ddfe829[collection1] main]
   [junit4]   2> 2101659 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[AFE15B4084CE5BF5]) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 2101660 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[AFE15B4084CE5BF5]) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 2101663 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[AFE15B4084CE5BF5]) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2101663 INFO  (searcherExecutor-8592-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 2101664 INFO  (searcherExecutor-8592-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 2101664 INFO  (searcherExecutor-8592-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 2101664 INFO  (searcherExecutor-8592-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 2101664 INFO  (searcherExecutor-8592-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 2101665 INFO  (searcherExecutor-8592-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 2101665 INFO  (searcherExecutor-8592-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 2101665 INFO  (searcherExecutor-8592-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 2101666 INFO  (searcherExecutor-8592-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 2101666 INFO  (searcherExecutor-8592-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3ddfe829[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2101666 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[AFE15B4084CE5BF5]) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=20, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.39186318344970494]
   [junit4]   2> 2101667 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[AFE15B4084CE5BF5]) [    x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 2101667 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[AFE15B4084CE5BF5]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6d5cf38[collection1] main]
   [junit4]   2> 2101668 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[AFE15B4084CE5BF5]) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@704d040
   [junit4]   2> 2101675 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[AFE15B4084CE5BF5]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[529 (1552236339175358464)]} 0 0
   [junit4]   2> 2101675 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[AFE15B4084CE5BF5]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:529&qt=standard&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 2101690 INFO  (searcherExecutor-8592-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6d5cf38[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2101927 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[AFE15B4084CE5BF5]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[530 (1552236339439599616)]} 0 252
   [junit4]   2> 2101927 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[AFE15B4084CE5BF5]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:530&qt=standard&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 2101927 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[AFE15B4084CE5BF5]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {delete=[529 (-1552236339440648192)]} 0 253
   [junit4]   2> 2101927 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[AFE15B4084CE5BF5]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:529&qt=standard&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 2101928 ERROR (TEST-HardAutoCommitTest.testCommitWithin-seed#[AFE15B4084CE5BF5]) [    x:collection1] o.a.s.SolrTestCaseJ4 REQUEST FAILED: xpath=//result[@numFound=1]
   [junit4]   2> 	xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]   2> <response>
   [junit4]   2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
   [junit4]   2> </response>
   [junit4]   2> 
   [junit4]   2> 	request was:q=id:529&qt=standard&start=0&rows=20&version=2.2
   [junit4]   2> 2101928 ERROR (TEST-HardAutoCommitTest.testCommitWithin-seed#[AFE15B4084CE5BF5]) [    x:collection1] o.a.s.SolrTestCaseJ4 REQUEST FAILED: q=id:529&qt=standard&start=0&rows=20&version=2.2:java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
   [junit4]   2> 	xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]   2> <response>
   [junit4]   2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
   [junit4]   2> </response>
   [junit4]   2> 
   [junit4]   2> 	request was:q=id:529&qt=standard&start=0&rows=20&version=2.2
   [junit4]   2> 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:811)
   [junit4]   2> 	at org.apache.solr.update.HardAutoCommitTest.testCommitWithin(HardAutoCommitTest.java:100)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:498)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 2101929 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[AFE15B4084CE5BF5]) [    x:collection1] o.a.s.SolrTestCaseJ4 ###Ending testCommitWithin
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=HardAutoCommitTest -Dtests.method=testCommitWithin -Dtests.seed=AFE15B4084CE5BF5 -Dtests.slow=true -Dtests.locale=id-ID -Dtests.timezone=Asia/Novosibirsk -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.54s J0 | HardAutoCommitTest.testCommitWithin <<<
   [junit4]    > Throwable #1: java.lang.RuntimeException: Exception during query
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([AFE15B4084CE5BF5:1533343807E0B5E0]:0)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:818)
   [junit4]    > 	at org.apache.solr.update.HardAutoCommitTest.testCommitWithin(HardAutoCommitTest.java:100)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]    > Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
   [junit4]    > 	xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]    > <response>
   [junit4]    > <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
   [junit4]    > </response>
   [junit4]    > 	request was:q=id:529&qt=standard&start=0&rows=20&version=2.2
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:811)
   [junit4]    > 	... 40 more
   [junit4]   2> 2101995 INFO  (SUITE-HardAutoCommitTest-seed#[AFE15B4084CE5BF5]-worker) [    x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> 2101995 INFO  (SUITE-HardAutoCommitTest-seed#[AFE15B4084CE5BF5]-worker) [    x:collection1] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1384133570
   [junit4]   2> 2101996 INFO  (coreCloseExecutor-8596-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@602ae270
   [junit4]   2> 2101996 INFO  (coreCloseExecutor-8596-thread-1-processing-x:collection1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter close.
   [junit4]   2> 2101996 INFO  (coreCloseExecutor-8596-thread-1-processing-x:collection1) [    x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@109e231c
   [junit4]   2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.update.HardAutoCommitTest_AFE15B4084CE5BF5-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {range_facet_l_dv=FST50, _version_=PostingsFormat(name=MockRandom), field_t=PostingsFormat(name=LuceneFixedGap), multiDefault=PostingsFormat(name=Memory), subject=PostingsFormat(name=Memory), intDefault=PostingsFormat(name=MockRandom), id=FST50, range_facet_i_dv=PostingsFormat(name=MockRandom), text=PostingsFormat(name=LuceneFixedGap), range_facet_l=PostingsFormat(name=MockRandom), timestamp=PostingsFormat(name=MockRandom)}, docValues:{range_facet_l_dv=DocValuesFormat(name=Direct), range_facet_i_dv=DocValuesFormat(name=Asserting), timestamp=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=160, maxMBSortInHeap=7.363303924623891, sim=RandomSimilarity(queryNorm=false): {}, locale=id-ID, timezone=Asia/Novosibirsk
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=3,threads=1,free=33512664,total=371486720
   [junit4]   2> NOTE: All tests run in this JVM: [TestReplicationHandler, BasicAuthIntegrationTest, TestFunctionQuery, XsltUpdateRequestHandlerTest, TestStressReorder, BooleanFieldTest, TermsComponentTest, TestSweetSpotSimilarityFactory, SolrPluginUtilsTest, TestRTGBase, AddBlockUpdateTest, TestExportWriter, BasicDistributedZkTest, TestLockTree, TestCustomSort, SuggestComponentTest, MigrateRouteKeyTest, TestLegacyFieldCache, TestFilteredDocIdSet, HighlighterTest, TestManagedSynonymFilterFactory, TestJsonFacets, TestShardHandlerFactory, TestCloudSchemaless, TestBM25SimilarityFactory, PeerSyncTest, StressHdfsTest, TestExactStatsCache, TestSolrQueryParserDefaultOperatorResource, UniqFieldsUpdateProcessorFactoryTest, TestPerFieldSimilarity, ReplicaListTransformerTest, TestNRTOpen, TestHighFrequencyDictionaryFactory, RequiredFieldsTest, DistributedFacetPivotSmallTest, TestCollapseQParserPlugin, SpatialFilterTest, TestReplicaProperties, TestMissingGroups, TestIBSimilarityFactory, DirectSolrSpellCheckerTest, TestPhraseSuggestions, TestRequestStatusCollectionAPI, AlternateDirectoryTest, ChaosMonkeyNothingIsSafeTest, PrimUtilsTest, TestLeaderInitiatedRecoveryThread, DistributedDebugComponentTest, TestSchemaVersionResource, CdcrRequestHandlerTest, TestFastLRUCache, TestRandomCollapseQParserPlugin, TestOverriddenPrefixQueryForCustomFieldType, TestSolrCloudSnapshots, DistributedFacetExistsSmallTest, TestManagedSchemaThreadSafety, OverseerRolesTest, SystemInfoHandlerTest, TestDistribIDF, SOLR749Test, ResponseHeaderTest, TestManagedSchema, TestPKIAuthenticationPlugin, TestUseDocValuesAsStored, PolyFieldTest, TestNumericTerms64, TestSolrJ, TestSolr4Spatial2, RecoveryZkTest, TestCollectionAPI, IndexSchemaRuntimeFieldTest, UUIDUpdateProcessorFallbackTest, DisMaxRequestHandlerTest, TestFileDictionaryLookup, TestPartialUpdateDeduplication, TestStandardQParsers, ZkNodePropsTest, TestReloadDeadlock, TestDocumentBuilder, SimpleFacetsTest, TestFieldCacheSort, TestFaceting, DistributedMLTComponentTest, SpatialRPTFieldTypeTest, DistributedQueueTest, TestRandomDVFaceting, TestCryptoKeys, DistributedQueryElevationComponentTest, OverseerTaskQueueTest, TestSimpleQParserPlugin, TestBinaryResponseWriter, TestElisionMultitermQuery, TestJettySolrRunner, TestRandomRequestDistribution, CopyFieldTest, TestRawTransformer, BigEndianAscendingWordSerializerTest, StatelessScriptUpdateProcessorFactoryTest, TestDistributedSearch, TestSolrDynamicMBean, TriLevelCompositeIdRoutingTest, TestSurroundQueryParser, TestAddFieldRealTimeGet, HdfsUnloadDistributedZkTest, TestRestManager, TestDocTermOrds, TestWordDelimiterFilterFactory, QueryResultKeyTest, HdfsCollectionsAPIDistributedZkTest, CheckHdfsIndexTest, AnalyticsMergeStrategyTest, PluginInfoTest, BaseCdcrDistributedZkTest, TestSolrConfigHandlerConcurrent, UnloadDistributedZkTest, CoreSorterTest, TestCSVLoader, TestSolrCloudWithSecureImpersonation, PingRequestHandlerTest, SolrCoreTest, CoreAdminRequestStatusTest, SparseHLLTest, CollectionStateFormat2Test, TestLFUCache, PKIAuthenticationIntegrationTest, MoreLikeThisHandlerTest, TestLRUCache, TestGroupingSearch, LeaderInitiatedRecoveryOnCommitTest, SearchHandlerTest, StandardRequestHandlerTest, ConnectionManagerTest, TestDynamicLoading, VersionInfoTest, OutputWriterTest, InfoHandlerTest, DistributedQueryComponentCustomSortTest, TestCharFilters, ReplaceNodeTest, TestObjectReleaseTracker, TestInfoStreamLogging, TestLeaderElectionWithEmptyReplica, EnumFieldTest, HLLSerializationTest, HdfsThreadLeakTest, ShardRoutingTest, ZkCLITest, SynonymTokenizerTest, DocValuesNotIndexedTest, ActionThrottleTest, TestStressLiveNodes, TestUniqueKeyFieldResource, DistributedSpellCheckComponentTest, SharedFSAutoReplicaFailoverUtilsTest, TestCloudPivotFacet, DirectUpdateHandlerTest, TestXmlQParserPlugin, SyncSliceTest, OverseerTest, LeaderElectionIntegrationTest, BasicZkTest, ClusterStateUpdateTest, TestRandomFaceting, LeaderElectionTest, ZkSolrClientTest, TestZkChroot, ShardRoutingCustomTest, AutoCommitTest, TestRecovery, TestHashPartitioner, ZkControllerTest, TestJoin, TestReload, TestStressVersions, HardAutoCommitTest]
   [junit4] Completed [275/658 (1!)] on J0 in 2.83s, 1 test, 1 error <<< FAILURES!

[...truncated 27 lines...]
   [junit4] Suite: org.apache.solr.core.TestLazyCores
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_AFE15B4084CE5BF5-001/init-core-data-001
   [junit4]   2> 2127991 INFO  (SUITE-TestLazyCores-seed#[AFE15B4084CE5BF5]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 2128000 INFO  (TEST-TestLazyCores.testRace-seed#[AFE15B4084CE5BF5]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testRace
   [junit4]   2> 2128023 INFO  (TEST-TestLazyCores.testRace-seed#[AFE15B4084CE5BF5]) [    ] o.a.s.c.SolrCores Allocating transient cache for 4 transient cores
   [junit4]   2> 2128038 INFO  (coreLoadExecutor-9049-thread-2) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2128043 INFO  (coreLoadExecutor-9049-thread-2) [    ] o.a.s.s.IndexSchema [collection2] Schema name=tiny
   [junit4]   2> 2128045 INFO  (coreLoadExecutor-9049-thread-2) [    ] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
   [junit4]   2> 2128047 INFO  (coreLoadExecutor-9049-thread-2) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection2' using configuration from instancedir /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_AFE15B4084CE5BF5-001/tempDir-001/collection2
   [junit4]   2> 2128047 INFO  (coreLoadExecutor-9049-thread-2) [    x:collection2] o.a.s.c.SolrCore [[collection2] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_AFE15B4084CE5BF5-001/tempDir-001/collection2], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_AFE15B4084CE5BF5-001/tempDir-001/collection2/data/]
   [junit4]   2> 2128052 INFO  (coreLoadExecutor-9049-thread-2) [    x:collection2] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=12, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 2128061 INFO  (coreLoadExecutor-9049-thread-2) [    x:collection2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2128061 INFO  (coreLoadExecutor-9049-thread-2) [    x:collection2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2128062 INFO  (coreLoadExecutor-9049-thread-2) [    x:collection2] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=16, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 2128065 INFO  (coreLoadExecutor-9049-thread-2) [    x:collection2] o.a.s.s.SolrIndexSearcher Opening [Searcher@516a0ca4[collection2] main]
   [junit4]   2> 2128065 INFO  (coreLoadExecutor-9049-thread-2) [    x:collection2] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_AFE15B4084CE5BF5-001/tempDir-001/collection2/conf
   [junit4]   2> 2128065 INFO  (coreLoadExecutor-9049-thread-2) [    x:collection2] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2128066 INFO  (coreLoadExecutor-9049-thread-2) [    x:collection2] o.a.s.c.SolrCores Opening transient core collection2
   [junit4]   2> 2128067 INFO  (coreLoadExecutor-9049-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2128092 INFO  (coreLoadExecutor-9049-thread-3) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2128093 INFO  (searcherExecutor-9050-thread-1-processing-x:collection2) [    x:collection2] o.a.s.c.SolrCore [collection2] Registered new searcher Searcher@516a0ca4[collection2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2128093 INFO  (coreLoadExecutor-9049-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=tiny
   [junit4]   2> 2128095 INFO  (coreLoadExecutor-9049-thread-1) [    ] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
   [junit4]   2> 2128097 INFO  (coreLoadExecutor-9049-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuratio

[...truncated too long message...]

ce/Lucene-Solr-master-Solaris/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/api-all-1.0.0-M20.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/bcprov-jdk15-1.45.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/cglib-nodep-2.2.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/commons-collections-3.2.2.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/commons-math3-3.4.1.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/easymock-3.0.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/ehcache-core-2.4.4.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/hadoop-common-2.7.2-tests.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/hadoop-hdfs-2.7.2-tests.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/hadoop-minikdc-2.7.2.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/jackson-annotations-2.5.4.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/jackson-databind-2.5.4.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/jersey-core-1.9.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/jersey-server-1.9.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/jetty-6.1.26.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/jetty-util-6.1.26.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/mina-core-2.0.0-M5.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/mockito-core-1.9.5.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/netty-all-4.0.36.Final.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/objenesis-1.2.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/presto-parser-0.122.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/test-lib/slice-0.10.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/build/analysis/icu/lucene-analyzers-icu-7.0.0-SNAPSHOT.jar:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/contrib/solr-analysis-extras/classes/java:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/build/queryparser/classes/test:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-launcher.jar:/export/home/jenkins/.ant/lib/ivy-2.3.0.jar:/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-logging.jar:/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jai.jar:/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-oro.jar:/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-swing.jar:/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-netrexx.jar:/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bcel.jar:/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-regexp.jar:/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bsf.jar:/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jsch.jar:/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit.jar:/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-antlr.jar:/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-net.jar:/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jmf.jar:/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant.jar:/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jdepend.jar:/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit4.jar:/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-javamail.jar:/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-xalan2.jar:/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-log4j.jar:/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-resolver.jar:/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-testutil.jar:/usr/jdk/instances/jdk1.8.0/lib/tools.jar:/export/home/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.4.0.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/temp/junit4-J1-20161128_093705_159.events @/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/temp/junit4-J1-20161128_093705_159.suites -stdin
   [junit4] ERROR: JVM J1 ended with an exception: java.io.IOException: No space left on device
   [junit4] 	at java.io.FileOutputStream.writeBytes(Native Method)
   [junit4] 	at java.io.FileOutputStream.write(FileOutputStream.java:326)
   [junit4] 	at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
   [junit4] 	at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
   [junit4] 	at java.io.FilterOutputStream.close(FilterOutputStream.java:158)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.io.Closer.close(Closer.java:214)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1488)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:765: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:709: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:59: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build.xml:252: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/common-build.xml:536: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:1443: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:984: At least one slave process threw an exception, first: No space left on device

Total time: 77 minutes 27 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 975 - Still Unstable!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/975/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseG1GC

5 tests failed.
FAILED:  org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.testDelegationTokenCancelFail

Error Message:
expected:<200> but was:<404>

Stack Trace:
java.lang.AssertionError: expected:<200> but was:<404>
	at __randomizedtesting.SeedInfo.seed([776A583AAFE19E9F:1FD56D107F7B8C73]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.junit.Assert.assertEquals(Assert.java:456)
	at org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.cancelDelegationToken(TestSolrCloudWithDelegationTokens.java:140)
	at org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.testDelegationTokenCancelFail(TestSolrCloudWithDelegationTokens.java:304)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testTwoUpdates

Error Message:
unknown UpdateRequestProcessorChain: <add><doc><field name="id">1</field><field name="text">The Apache Software Foundation is happy to announce BarCampApache Sydney, Australia, the first ASF-backed event in the Southern Hemisphere!</field></doc></add>

Stack Trace:
org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">1</field><field name="text">The Apache Software Foundation is happy to announce BarCampApache Sydney, Australia, the first ASF-backed event in the Southern Hemisphere!</field></doc></add>
	at __randomizedtesting.SeedInfo.seed([1AE011E8E88BC1C9:17AA46B7F0A4A367]:0)
	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testTwoUpdates(UIMAUpdateRequestProcessorTest.java:101)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing

Error Message:
unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes.    Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>

Stack Trace:
org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
  Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
	at __randomizedtesting.SeedInfo.seed([1AE011E8E88BC1C9:96E159AAF784063]:0)
	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing(UIMAUpdateRequestProcessorTest.java:83)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testMultiplierProcessing

Error Message:
unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes.    Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>

Stack Trace:
org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
  Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
	at __randomizedtesting.SeedInfo.seed([1AE011E8E88BC1C9:21E1DB45EFDFED3D]:0)
	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing(UIMAUpdateRequestProcessorTest.java:83)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testMultiplierProcessing(UIMAUpdateRequestProcessorTest.java:177)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testErrorHandling

Error Message:
unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes.    Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>

Stack Trace:
org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
  Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
	at __randomizedtesting.SeedInfo.seed([1AE011E8E88BC1C9:2F146FD6BE5D401D]:0)
	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testErrorHandling(UIMAUpdateRequestProcessorTest.java:137)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 12204 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestSolrCloudWithDelegationTokens
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithDelegationTokens_776A583AAFE19E9F-001/init-core-data-001
   [junit4]   2> 2285914 INFO  (SUITE-TestSolrCloudWithDelegationTokens-seed#[776A583AAFE19E9F]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 2285918 INFO  (SUITE-TestSolrCloudWithDelegationTokens-seed#[776A583AAFE19E9F]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithDelegationTokens_776A583AAFE19E9F-001/tempDir-001
   [junit4]   2> 2285918 INFO  (SUITE-TestSolrCloudWithDelegationTokens-seed#[776A583AAFE19E9F]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2285919 INFO  (Thread-5036) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2285919 INFO  (Thread-5036) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2286019 INFO  (SUITE-TestSolrCloudWithDelegationTokens-seed#[776A583AAFE19E9F]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:45568
   [junit4]   2> 2286031 INFO  (jetty-launcher-2826-thread-1) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2286032 INFO  (jetty-launcher-2826-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1c8bcded{/solr,null,AVAILABLE}
   [junit4]   2> 2286035 INFO  (jetty-launcher-2826-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@aef18d8{SSL,[ssl, http/1.1]}{127.0.0.1:55832}
   [junit4]   2> 2286035 INFO  (jetty-launcher-2826-thread-1) [    ] o.e.j.s.Server Started @2289953ms
   [junit4]   2> 2286035 INFO  (jetty-launcher-2826-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=55832}
   [junit4]   2> 2286036 ERROR (jetty-launcher-2826-thread-1) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2286036 INFO  (jetty-launcher-2826-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solrâ„¢ version 7.0.0
   [junit4]   2> 2286037 INFO  (jetty-launcher-2826-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2286037 INFO  (jetty-launcher-2826-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2286037 INFO  (jetty-launcher-2826-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2016-11-27T23:58:40.035Z
   [junit4]   2> 2286039 INFO  (jetty-launcher-2826-thread-2) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2286040 INFO  (jetty-launcher-2826-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7d08122e{/solr,null,AVAILABLE}
   [junit4]   2> 2286042 INFO  (jetty-launcher-2826-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@47e60480{SSL,[ssl, http/1.1]}{127.0.0.1:52241}
   [junit4]   2> 2286042 INFO  (jetty-launcher-2826-thread-2) [    ] o.e.j.s.Server Started @2289960ms
   [junit4]   2> 2286042 INFO  (jetty-launcher-2826-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=52241}
   [junit4]   2> 2286042 ERROR (jetty-launcher-2826-thread-2) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2286042 INFO  (jetty-launcher-2826-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solrâ„¢ version 7.0.0
   [junit4]   2> 2286042 INFO  (jetty-launcher-2826-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2286042 INFO  (jetty-launcher-2826-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2286043 INFO  (jetty-launcher-2826-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2016-11-27T23:58:40.040Z
   [junit4]   2> 2286050 INFO  (jetty-launcher-2826-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2286058 INFO  (jetty-launcher-2826-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45568/solr
   [junit4]   2> 2286061 INFO  (jetty-launcher-2826-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2286067 INFO  (jetty-launcher-2826-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45568/solr
   [junit4]   2> 2286096 INFO  (jetty-launcher-2826-thread-1) [n:127.0.0.1:55832_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:55832_solr
   [junit4]   2> 2286098 INFO  (jetty-launcher-2826-thread-1) [n:127.0.0.1:55832_solr    ] o.a.s.c.Overseer Overseer (id=97012358835011588-127.0.0.1:55832_solr-n_0000000000) starting
   [junit4]   2> 2286110 INFO  (jetty-launcher-2826-thread-1) [n:127.0.0.1:55832_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55832_solr
   [junit4]   2> 2286113 INFO  (zkCallback-2836-thread-1-processing-n:127.0.0.1:55832_solr) [n:127.0.0.1:55832_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2286114 INFO  (jetty-launcher-2826-thread-2) [n:127.0.0.1:52241_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2286121 INFO  (jetty-launcher-2826-thread-2) [n:127.0.0.1:52241_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52241_solr
   [junit4]   2> 2286142 INFO  (zkCallback-2836-thread-2-processing-n:127.0.0.1:55832_solr) [n:127.0.0.1:55832_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2286159 INFO  (jetty-launcher-2826-thread-2) [n:127.0.0.1:52241_solr    ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HttpParamDelegationTokenPlugin
   [junit4]   2> 2286171 INFO  (jetty-launcher-2826-thread-2) [n:127.0.0.1:52241_solr    ] o.a.s.s.KerberosPlugin Params: {token.valid=30, delegation-token.token-kind=solr-dt, cookie.domain=127.0.0.1, cookie.path=/, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, token.validity=36000, signer.secret.provider.zookeeper.path=/token, type=kerberos, zk-dt-secret-manager.enable=true, kerberos.name.rules=DEFAULT, signer.secret.provider=zookeeper}
   [junit4]   2> 2286242 INFO  (zkCallback-2838-thread-1-processing-n:127.0.0.1:52241_solr) [n:127.0.0.1:52241_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2286302 INFO  (jetty-launcher-2826-thread-1) [n:127.0.0.1:55832_solr    ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HttpParamDelegationTokenPlugin
   [junit4]   2> 2286302 INFO  (jetty-launcher-2826-thread-1) [n:127.0.0.1:55832_solr    ] o.a.s.s.KerberosPlugin Params: {token.valid=30, delegation-token.token-kind=solr-dt, cookie.domain=127.0.0.1, cookie.path=/, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, token.validity=36000, signer.secret.provider.zookeeper.path=/token, type=kerberos, zk-dt-secret-manager.enable=true, kerberos.name.rules=DEFAULT, signer.secret.provider=zookeeper}
   [junit4]   2> 2286324 INFO  (jetty-launcher-2826-thread-1) [n:127.0.0.1:55832_solr    ] o.a.c.f.i.CuratorFrameworkImpl Starting
   [junit4]   2> 2286324 INFO  (jetty-launcher-2826-thread-2) [n:127.0.0.1:52241_solr    ] o.a.c.f.i.CuratorFrameworkImpl Starting
   [junit4]   2> 2286352 INFO  (jetty-launcher-2826-thread-2-EventThread) [n:127.0.0.1:52241_solr    ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
   [junit4]   2> 2286357 INFO  (jetty-launcher-2826-thread-1-EventThread) [n:127.0.0.1:55832_solr    ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
   [junit4]   2> 2286514 INFO  (jetty-launcher-2826-thread-2) [n:127.0.0.1:52241_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithDelegationTokens_776A583AAFE19E9F-001/tempDir-001/node2/.
   [junit4]   2> 2286544 INFO  (jetty-launcher-2826-thread-1) [n:127.0.0.1:55832_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithDelegationTokens_776A583AAFE19E9F-001/tempDir-001/node1/.
   [junit4]   2> 2286578 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenSolrClient-seed#[776A583AAFE19E9F]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testDelegationTokenSolrClient
   [junit4]   2> 2286672 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenSolrClient-seed#[776A583AAFE19E9F]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=  , 01-I-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 01-I-1970 00:00:00 GMT
   [junit4]   2> 2286693 INFO  (qtp710432741-19114) [n:127.0.0.1:52241_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&delegation=GgADYmFyA2JhcgCKAVioOfXmigFYzEZ55gECFNmOyVr6orEXQmRfNsHSoAqHzDGjB3NvbHItZHQA&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2286693 INFO  (qtp710432741-19114) [n:127.0.0.1:52241_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&delegation=GgADYmFyA2JhcgCKAVioOfXmigFYzEZ55gECFNmOyVr6orEXQmRfNsHSoAqHzDGjB3NvbHItZHQA&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2286694 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenSolrClient-seed#[776A583AAFE19E9F]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testDelegationTokenSolrClient
   [junit4]   2> 2286715 INFO  (TEST-TestSolrCloudWithDelegationTokens.testZNodePaths-seed#[776A583AAFE19E9F]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testZNodePaths
   [junit4]   2> 2286727 INFO  (TEST-TestSolrCloudWithDelegationTokens.testZNodePaths-seed#[776A583AAFE19E9F]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testZNodePaths
   [junit4]   2> 2286765 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancelFail-seed#[776A583AAFE19E9F]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testDelegationTokenCancelFail
   [junit4]   2> 2286817 ERROR (qtp177340620-19091) [    ] o.a.h.s.t.d.ZKDelegationTokenSecretManager No node in path [/ZKDTSMTokensRoot/DT_3]
   [junit4]   2> 2286819 ERROR (qtp710432741-19107) [    ] o.a.h.s.t.d.ZKDelegationTokenSecretManager No node in path [/ZKDTSMTokensRoot/DT_3]
   [junit4]   2> 2286856 ERROR (qtp177340620-19096) [    ] o.a.h.s.t.d.ZKDelegationTokenSecretManager No node in path [/ZKDTSMTokensRoot/DT_4]
   [junit4]   2> 2286863 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancelFail-seed#[776A583AAFE19E9F]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testDelegationTokenCancelFail
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestSolrCloudWithDelegationTokens -Dtests.method=testDelegationTokenCancelFail -Dtests.seed=776A583AAFE19E9F -Dtests.slow=true -Dtests.locale=bg -Dtests.timezone=Australia/Canberra -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 0.14s J0 | TestSolrCloudWithDelegationTokens.testDelegationTokenCancelFail <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: expected:<200> but was:<404>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([776A583AAFE19E9F:1FD56D107F7B8C73]:0)
   [junit4]    > 	at org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.cancelDelegationToken(TestSolrCloudWithDelegationTokens.java:140)
   [junit4]    > 	at org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.testDelegationTokenCancelFail(TestSolrCloudWithDelegationTokens.java:304)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2286902 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationOtherOp-seed#[776A583AAFE19E9F]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testDelegationOtherOp
   [junit4]   2> 2286909 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationOtherOp-seed#[776A583AAFE19E9F]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2286911 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationOtherOp-seed#[776A583AAFE19E9F]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45568/solr ready
   [junit4]   2> 2286916 INFO  (qtp710432741-19108) [n:127.0.0.1:52241_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params op=someSolrOperation&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2286916 INFO  (qtp710432741-19108) [n:127.0.0.1:52241_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={op=someSolrOperation&action=LIST&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2286917 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationOtherOp-seed#[776A583AAFE19E9F]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=bar&p=bar&t=test&e=1480327120914&s=fcbJa+KF+vUFjUJ7hHsgHXAui28="; Path=/; Domain=127.0.0.1; Expires=  , 28-XI-2016 09:58:40 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 28-XI-2016 09:58:40 GMT
   [junit4]   2> 2286918 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationOtherOp-seed#[776A583AAFE19E9F]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testDelegationOtherOp
   [junit4]   2> 2286944 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancel-seed#[776A583AAFE19E9F]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testDelegationTokenCancel
   [junit4]   2> 2286964 ERROR (qtp710432741-19101) [    ] o.a.h.s.t.d.ZKDelegationTokenSecretManager No node in path [/ZKDTSMTokensRoot/DT_5]
   [junit4]   2> 2287008 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancel-seed#[776A583AAFE19E9F]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=  , 01-I-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 01-I-1970 00:00:00 GMT
   [junit4]   2> 2287016 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancel-seed#[776A583AAFE19E9F]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=  , 01-I-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 01-I-1970 00:00:00 GMT
   [junit4]   2> 2287036 ERROR (qtp177340620-19091) [    ] o.a.h.s.t.d.ZKDelegationTokenSecretManager No node in path [/ZKDTSMTokensRoot/DT_6]
   [junit4]   2> 2287047 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancel-seed#[776A583AAFE19E9F]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=  , 01-I-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 01-I-1970 00:00:00 GMT
   [junit4]   2> 2287052 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancel-seed#[776A583AAFE19E9F]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2287053 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancel-seed#[776A583AAFE19E9F]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45568/solr ready
   [junit4]   2> 2287060 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancel-seed#[776A583AAFE19E9F]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=  , 01-I-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 01-I-1970 00:00:00 GMT
   [junit4]   2> 2287061 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancel-seed#[776A583AAFE19E9F]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testDelegationTokenCancel
   [junit4]   2> 2287090 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[776A583AAFE19E9F]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testDelegationTokenRenew
   [junit4]   2> 2287119 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[776A583AAFE19E9F]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2287120 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[776A583AAFE19E9F]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45568/solr ready
   [junit4]   2> 2287124 INFO  (qtp710432741-19103) [n:127.0.0.1:52241_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&delegation=GgADYmFyA2JhcgCKAVioOffAigFYzEZ7wAcCFP6SZRNJrjQo-gFyii_a08ZijvxUB3NvbHItZHQA&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2287125 INFO  (qtp710432741-19103) [n:127.0.0.1:52241_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&delegation=GgADYmFyA2JhcgCKAVioOffAigFYzEZ7wAcCFP6SZRNJrjQo-gFyii_a08ZijvxUB3NvbHItZHQA&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2287132 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[776A583AAFE19E9F]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=  , 01-I-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 01-I-1970 00:00:00 GMT
   [junit4]   2> 2287138 INFO  (qtp177340620-19093) [n:127.0.0.1:55832_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&delegation=GgADYmFyA2JhcgCKAVioOffAigFYzEZ7wAcCFP6SZRNJrjQo-gFyii_a08ZijvxUB3NvbHItZHQA&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2287138 INFO  (qtp177340620-19093) [n:127.0.0.1:55832_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&delegation=GgADYmFyA2JhcgCKAVioOffAigFYzEZ7wAcCFP6SZRNJrjQo-gFyii_a08ZijvxUB3NvbHItZHQA&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2287145 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[776A583AAFE19E9F]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=  , 01-I-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 01-I-1970 00:00:00 GMT
   [junit4]   2> 2287167 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[776A583AAFE19E9F]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2287169 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[776A583AAFE19E9F]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45568/solr ready
   [junit4]   2> 2287173 INFO  (qtp710432741-19106) [n:127.0.0.1:52241_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&delegation=GgADYmFyA2JhcgCKAVioOff4igFYzEZ7-AgCFDU-X5y9SrPw36snHzDxe6U4zuGRB3NvbHItZHQA&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2287173 INFO  (qtp710432741-19106) [n:127.0.0.1:52241_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&delegation=GgADYmFyA2JhcgCKAVioOff4igFYzEZ7-AgCFDU-X5y9SrPw36snHzDxe6U4zuGRB3NvbHItZHQA&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2287179 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[776A583AAFE19E9F]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2287181 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[776A583AAFE19E9F]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45568/solr ready
   [junit4]   2> 2287185 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[776A583AAFE19E9F]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=  , 01-I-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 01-I-1970 00:00:00 GMT
   [junit4]   2> 2287191 INFO  (qtp177340620-19087) [n:127.0.0.1:55832_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&delegation=GgADYmFyA2JhcgCKAVioOff4igFYzEZ7-AgCFDU-X5y9SrPw36snHzDxe6U4zuGRB3NvbHItZHQA&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2287191 INFO  (qtp177340620-19087) [n:127.0.0.1:55832_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&delegation=GgADYmFyA2JhcgCKAVioOff4igFYzEZ7-AgCFDU-X5y9SrPw36snHzDxe6U4zuGRB3NvbHItZHQA&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2287196 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[776A583AAFE19E9F]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2287198 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[776A583AAFE19E9F]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45568/solr ready
   [junit4]   2> 2287202 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[776A583AAFE19E9F]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=  , 01-I-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 01-I-1970 00:00:00 GMT
   [junit4]   2> 2287218 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[776A583AAFE19E9F]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2287220 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[776A583AAFE19E9F]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45568/solr ready
   [junit4]   2> 2287224 INFO  (qtp710432741-19107) [n:127.0.0.1:52241_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&delegation=GgADYmFyA2JhcgCKAVioOfgzigFYzEZ8MwkCFE3y0prfM-nNctWf4PAmhcW0cRbNB3NvbHItZHQA&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2287224 INFO  (qtp710432741-19107) [n:127.0.0.1:52241_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&delegation=GgADYmFyA2JhcgCKAVioOfgzigFYzEZ8MwkCFE3y0prfM-nNctWf4PAmhcW0cRbNB3NvbHItZHQA&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2287231 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[776A583AAFE19E9F]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2287232 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[776A583AAFE19E9F]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45568/solr ready
   [junit4]   2> 2287236 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[776A583AAFE19E9F]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=  , 01-I-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 01-I-1970 00:00:00 GMT
   [junit4]   2> 2287242 INFO  (qtp177340620-19091) [n:127.0.0.1:55832_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&delegation=GgADYmFyA2JhcgCKAVioOfgzigFYzEZ8MwkCFE3y0prfM-nNctWf4PAmhcW0cRbNB3NvbHItZHQA&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2287242 INFO  (qtp177340620-19091) [n:127.0.0.1:55832_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&delegation=GgADYmFyA2JhcgCKAVioOfgzigFYzEZ8MwkCFE3y0prfM-nNctWf4PAmhcW0cRbNB3NvbHItZHQA&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2287247 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[776A583AAFE19E9F]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=  , 01-I-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 01-I-1970 00:00:00 GMT
   [junit4]   2> 2287262 INFO  (qtp710432741-19106) [n:127.0.0.1:52241_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&delegation=GgADYmFyA2JhcgCKAVioOfheigFYzEZ8XgoCFK65Xi2GNt9YRqrk5vofJDypqlXrB3NvbHItZHQA&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2287262 INFO  (qtp710432741-19106) [n:127.0.0.1:52241_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&delegation=GgADYmFyA2JhcgCKAVioOfheigFYzEZ8XgoCFK65Xi2GNt9YRqrk5vofJDypqlXrB3NvbHItZHQA&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2287266 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[776A583AAFE19E9F]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=  , 01-I-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 01-I-1970 00:00:00 GMT
   [junit4]   2> 2287271 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[776A583AAFE19E9F]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2287273 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[776A583AAFE19E9F]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45568/solr ready
   [junit4]   2> 2287277 INFO  (qtp710432741-19103) [n:127.0.0.1:52241_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&delegation=GgADYmFyA2JhcgCKAVioOfheigFYzEZ8XgoCFK65Xi2GNt9YRqrk5vofJDypqlXrB3NvbHItZHQA&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2287277 INFO  (qtp710432741-19103) [n:127.0.0.1:52241_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&delegation=GgADYmFyA2JhcgCKAVioOfheigFYzEZ8XgoCFK65Xi2GNt9YRqrk5vofJDypqlXrB3NvbHItZHQA&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2287284 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[776A583AAFE19E9F]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=  , 01-I-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 01-I-1970 00:00:00 GMT
   [junit4]   2> 2287284 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[776A583AAFE19E9F]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testDelegationTokenRenew
   [junit4]   2> 2287309 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenewFail-seed#[776A583AAFE19E9F]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testDelegationTokenRenewFail
   [junit4]   2> 2287315 WARN  (qtp710432741-19102) [    ] o.a.h.s.a.s.AuthenticationFilter Authentication exception: org.apache.hadoop.security.AccessControlException: foo tries to renew a token with renewer bar
   [junit4]   2> 2287315 WARN  (qtp710432741-19102) [    ] o.a.s.s.KerberosPlugin Could not find org.apache.solr.security.kerberosplugin.requestcontinues
   [junit4]   2> 2287315 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenewFail-seed#[776A583AAFE19E9F]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=  , 01-I-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 01-I-1970 00:00:00 GMT
   [junit4]   2> 2287316 WARN  (qtp177340620-19094) [    ] o.a.h.s.a.s.AuthenticationFilter Authentication exception: org.apache.hadoop.security.AccessControlException: foo tries to renew a token with renewer bar
   [junit4]   2> 2287317 WARN  (qtp177340620-19094) [    ] o.a.s.s.KerberosPlugin Could not find org.apache.solr.security.kerberosplugin.requestcontinues
   [junit4]   2> 2287318 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenewFail-seed#[776A583AAFE19E9F]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=  , 01-I-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 01-I-1970 00:00:00 GMT
   [junit4]   2> 2287325 WARN  (qtp710432741-19113) [    ] o.a.h.s.a.s.AuthenticationFilter Authentication exception: org.apache.hadoop.security.AccessControlException: notRenewUser tries to renew a token with renewer renewUser
   [junit4]   2> 2287326 WARN  (qtp710432741-19113) [    ] o.a.s.s.KerberosPlugin Could not find org.apache.solr.security.kerberosplugin.requestcontinues
   [junit4]   2> 2287326 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenewFail-seed#[776A583AAFE19E9F]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=  , 01-I-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 01-I-1970 00:00:00 GMT
   [junit4]   2> 2287327 WARN  (qtp177340620-19093) [n:127.0.0.1:55832_solr    ] o.a.h.s.a.s.AuthenticationFilter Authentication exception: org.apache.hadoop.security.AccessControlException: notRenewUser tries to renew a token with renewer renewUser
   [junit4]   2> 2287327 WARN  (qtp177340620-19093) [n:127.0.0.1:55832_solr    ] o.a.s.s.KerberosPlugin Could not find org.apache.solr.security.kerberosplugin.requestcontinues
   [junit4]   2> 2287328 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenewFail-seed#[776A583AAFE19E9F]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=  , 01-I-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 01-I-1970 00:00:00 GMT
   [junit4]   2> 2287328 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenewFail-seed#[776A583AAFE19E9F]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testDelegationTokenRenewFail
   [junit4]   2> 2287357 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenVerify-seed#[776A583AAFE19E9F]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testDelegationTokenVerify
   [junit4]   2> 2287368 INFO  (qtp710432741-19106) [n:127.0.0.1:52241_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&delegation=GgADYmFyA2JhcgCKAVioOfjMigFYzEZ8zA0CFHtBr5AuHg8_4XKPq6-AQpae5EOOB3NvbHItZHQA&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2287369 INFO  (qtp710432741-19106) [n:127.0.0.1:52241_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&delegation=GgADYmFyA2JhcgCKAVioOfjMigFYzEZ8zA0CFHtBr5AuHg8_4XKPq6-AQpae5EOOB3NvbHItZHQA&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2287375 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenVerify-seed#[776A583AAFE19E9F]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=  , 01-I-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 01-I-1970 00:00:00 GMT
   [junit4]   2> 2287379 INFO  (qtp177340620-19089) [n:127.0.0.1:55832_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&delegation=GgADYmFyA2JhcgCKAVioOfjMigFYzEZ8zA0CFHtBr5AuHg8_4XKPq6-AQpae5EOOB3NvbHItZHQA&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2287379 INFO  (qtp177340620-19089) [n:127.0.0.1:55832_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&delegation=GgADYmFyA2JhcgCKAVioOfjMigFYzEZ8zA0CFHtBr5AuHg8_4XKPq6-AQpae5EOOB3NvbHItZHQA&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2287384 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenVerify-seed#[776A583AAFE19E9F]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2287386 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenVerify-seed#[776A583AAFE19E9F]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45568/solr ready
   [junit4]   2> 2287390 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenVerify-seed#[776A583AAFE19E9F]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=  , 01-I-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 01-I-1970 00:00:00 GMT
   [junit4]   2> 2287391 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenVerify-seed#[776A583AAFE19E9F]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testDelegationTokenVerify
   [junit4]   2> 2287392 INFO  (jetty-launcher-2826-thread-1) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@47e60480{SSL,[ssl, http/1.1]}{127.0.0.1:0}
   [junit4]   2> 2287392 INFO  (jetty-launcher-2826-thread-2) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@aef18d8{SSL,[ssl, http/1.1]}{127.0.0.1:0}
   [junit4]   2> 2287393 INFO  (jetty-launcher-2826-thread-1) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=530591910
   [junit4]   2> 2287394 INFO  (jetty-launcher-2826-thread-2) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=2039498474
   [junit4]   2> 2287399 INFO  (jetty-launcher-2826-thread-2) [    ] o.a.s.c.Overseer Overseer (id=97012358835011588-127.0.0.1:55832_solr-n_0000000000) closing
   [junit4]   2> 2287402 ERROR (Thread[Thread-5059,5,TGRP-TestSolrCloudWithDelegationTokens]) [n:127.0.0.1:52241_solr    ] o.a.h.s.t.d.AbstractDelegationTokenSecretManager ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 2287404 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x158a839f32f0006, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2287405 INFO  (OverseerStateUpdate-97012358835011588-127.0.0.1:55832_solr-n_0000000000) [n:127.0.0.1:55832_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:55832_solr
   [junit4]   2> 2287412 INFO  (jetty-launcher-2826-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@7d08122e{/solr,null,UNAVAILABLE}
   [junit4]   2> 2288901 WARN  (zkCallback-2836-thread-2-processing-n:127.0.0.1:55832_solr) [n:127.0.0.1:55832_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 2288901 ERROR (Thread[Thread-5062,5,TGRP-TestSolrCloudWithDelegationTokens]) [n:127.0.0.1:55832_solr    ] o.a.h.s.t.d.AbstractDelegationTokenSecretManager ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 2288905 INFO  (jetty-launcher-2826-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1c8bcded{/solr,null,UNAVAILABLE}
   [junit4]   2> 2288908 INFO  (SUITE-TestSolrCloudWithDelegationTokens-seed#[776A583AAFE19E9F]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:45568 45568
   [junit4]   2> 2289054 INFO  (Thread-5036) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:45568 45568
   [junit4]   2> 2289056 WARN  (Thread-5036) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	12	/solr/aliases.json
   [junit4]   2> 	2	/solr/security.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	12	/solr/clusterprops.json
   [junit4]   2> 	12	/solr/clusterstate.json
   [junit4]   2> 	4	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMKeyIdRoot
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_7
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_6
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_9
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_8
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_1
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_1
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_3
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_3
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_2
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_2
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_5
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_4
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_4
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_13
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_12
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMSeqNumRoot
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_11
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_10
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	12	/solr/live_nodes
   [junit4]   2> 	12	/solr/collections
   [junit4]   2> 	3	/solr/overseer/queue
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot
   [junit4]   2> 	2	/solr/overseer/queue-work
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot
   [junit4]   2> 
   [junit4]   2> 2289057 INFO  (SUITE-TestSolrCloudWithDelegationTokens-seed#[776A583AAFE19E9F]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithDelegationTokens_776A583AAFE19E9F-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=665, maxMBSortInHeap=5.045842941191898, sim=RandomSimilarity(queryNorm=false): {}, locale=bg, timezone=Australia/Canberra
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=3,threads=1,free=329044928,total=536870912
   [junit4]   2> NOTE: All tests run in this JVM: [UnloadDistributedZkTest, DocValuesNotIndexedTest, TestFunctionQuery, TestDocTermOrds, AssignTest, CursorMarkTest, TestFastLRUCache, TestSolrJ, LeaderFailoverAfterPartitionTest, SpatialRPTFieldTypeTest, PingRequestHandlerTest, DateFieldTest, FileUtilsTest, DocValuesMultiTest, ClassificationUpdateProcessorIntegrationTest, RecoveryZkTest, TestPartialUpdateDeduplication, DirectUpdateHandlerTest, DateRangeFieldTest, TimeZoneUtilsTest, CloudMLTQParserTest, OverseerTest, TestLeaderInitiatedRecoveryThread, SuggestComponentTest, TestSort, TestWordDelimiterFilterFactory, DistributedFacetPivotLongTailTest, TestOnReconnectListenerSupport, TestFastOutputStream, HdfsThreadLeakTest, OpenExchangeRatesOrgProviderTest, TestCharFilters, OverseerCollectionConfigSetProcessorTest, TestSchemaVersionResource, SpatialFilterTest, TriLevelCompositeIdRoutingTest, BlockJoinFacetRandomTest, SystemInfoHandlerTest, SparseHLLTest, FastVectorHighlighterTest, TestIBSimilarityFactory, AtomicUpdatesTest, LeaderElectionTest, DistributedQueryElevationComponentTest, SolrCoreTest, TestNumericTerms64, SSLMigrationTest, TestFieldResource, TestPhraseSuggestions, TestLegacyNumericRangeQueryBuilder, TestCloudPivotFacet, TestPushWriter, TestSimpleQParserPlugin, TestShortCircuitedRequests, TestCollapseQParserPlugin, TestOrdValues, TestExactStatsCache, CopyFieldTest, TestExpandComponent, TestSlowCompositeReaderWrapper, TestManagedResource, DistributedMLTComponentTest, SolrCLIZkUtilsTest, TestManagedSynonymFilterFactory, TestDistributedSearch, TestLFUCache, TestSha256AuthenticationProvider, StressHdfsTest, DeleteStatusTest, TestClassicSimilarityFactory, MigrateRouteKeyTest, TestJettySolrRunner, TestSolrConfigHandlerConcurrent, SegmentsInfoRequestHandlerTest, BasicDistributedZkTest, TestXmlQParserPlugin, DistributedFacetPivotSmallTest, ResponseHeaderTest, BooleanFieldTest, TestLRUCache, HdfsDirectoryTest, TestFieldCacheVsDocValues, HttpPartitionTest, TestPostingsSolrHighlighter, DisMaxRequestHandlerTest, TestLazyCores, RestartWhileUpdatingTest, TestRestoreCore, PreAnalyzedUpdateProcessorTest, SolrXmlInZkTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, StatelessScriptUpdateProcessorFactoryTest, ShardRoutingTest, CustomCollectionTest, HardAutoCommitTest, TestLeaderElectionWithEmptyReplica, TestStressVersions, ConnectionManagerTest, HdfsRestartWhileUpdatingTest, ZkControllerTest, TestBM25SimilarityFactory, TestCloudDeleteByQuery, JSONWriterTest, HdfsRecoverLeaseTest, ZkCLITest, PathHierarchyTokenizerFactoryTest, TermsComponentTest, TestExclusionRuleCollectionAccess, RankQueryTest, ExplicitHLLTest, SchemaVersionSpecificBehaviorTest, VersionInfoTest, SynonymTokenizerTest, EnumFieldTest, CloneFieldUpdateProcessorFactoryTest, ShardSplitTest, XmlUpdateRequestHandlerTest, SyncSliceTest, OverseerRolesTest, OutputWriterTest, HdfsUnloadDistributedZkTest, ZkStateReaderTest, SearchHandlerTest, TestFileDictionaryLookup, RecoveryAfterSoftCommitTest, SolrPluginUtilsTest, TestBinaryField, TestSolrDynamicMBean, TestFieldCacheSort, TestFieldCacheReopen, TestSubQueryTransformerCrossCore, TestMergePolicyConfig, ConcurrentDeleteAndCreateCollectionTest, SOLR749Test, CdcrUpdateLogTest, BinaryUpdateRequestHandlerTest, ExitableDirectoryReaderTest, TestRawTransformer, SolrRequestParserTest, TestBinaryResponseWriter, TestXIncludeConfig, DistributedQueryComponentCustomSortTest, CdcrRequestHandlerTest, TestStandardQParsers, TestDocumentBuilder, TestExtendedDismaxParser, TestChildDocTransformer, TestRandomDVFaceting, TestRandomRequestDistribution, TestHashPartitioner, PKIAuthenticationIntegrationTest, TestRestManager, SimplePostToolTest, TestDistribIDF, SimpleFacetsTest, TestCursorMarkWithoutUniqueKey, TestHighlightDedupGrouping, TestSimpleTrackingShardHandler, ActionThrottleTest, BaseCdcrDistributedZkTest, CdcrBootstrapTest, CollectionsAPIAsyncDistributedZkTest, ConfigSetsAPITest, CreateCollectionCleanupTest, DeleteLastCustomShardedReplicaTest, DistribDocExpirationUpdateProcessorTest, ForceLeaderTest, PeerSyncReplicationTest, TestClusterProperties, TestCollectionAPI, TestDeleteCollectionOnDownNodes, TestDistribDocBasedVersion, TestLeaderElectionZkExpiry, TestLockTree, TestMiniSolrCloudCluster, TestRequestForwarding, TestRequestStatusCollectionAPI, TestSolrCloudWithDelegationTokens]
   [junit4] Completed [514/658 (1!)] on J0 in 3.16s, 8 tests, 1 failure <<< FAILURES!

[...truncated 11616 lines...]
   [junit4] Suite: org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/contrib/solr-uima/test/J0/temp/solr.uima.processor.UIMAUpdateRequestProcessorTest_1AE011E8E88BC1C9-001/init-core-data-001
   [junit4]   2> log4j:WARN No appenders could be found for logger (org.eclipse.jetty.util.log).
   [junit4]   2> log4j:WARN Please initialize the log4j system properly.
   [junit4]   2> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
   [junit4]   2> nov. 28, 2016 11:15:21 AM WhitespaceTokenizer initialize
   [junit4]   2> INFO: "Whitespace tokenizer successfully initialized"
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=UIMAUpdateRequestProcessorTest -Dtests.method=testTwoUpdates -Dtests.seed=1AE011E8E88BC1C9 -Dtests.slow=true -Dtests.locale=lv -Dtests.timezone=Australia/Canberra -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.02s J0 | UIMAUpdateRequestProcessorTest.testTwoUpdates <<<
   [junit4]    > Throwable #1: org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">1</field><field name="text">The Apache Software Foundation is happy to announce BarCampApache Sydney, Australia, the first ASF-backed event in the Southern Hemisphere!</field></doc></add>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([1AE011E8E88BC1C9:17AA46B7F0A4A367]:0)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
   [junit4]    > 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testTwoUpdates(UIMAUpdateRequestProcessorTest.java:101)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> nov. 28, 2016 11:15:25 AM WhitespaceTokenizer initialize
   [junit4]   2> INFO: "Whitespace tokenizer successfully initialized"
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=UIMAUpdateRequestProcessorTest -Dtests.method=testProcessing -Dtests.seed=1AE011E8E88BC1C9 -Dtests.slow=true -Dtests.locale=lv -Dtests.timezone=Australia/Canberra -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.01s J0 | UIMAUpdateRequestProcessorTest.testProcessing <<<
   [junit4]    > Throwable #1: org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
   [junit4]    >   Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([1AE011E8E88BC1C9:96E159AAF784063]:0)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
   [junit4]    > 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing(UIMAUpdateRequestProcessorTest.java:83)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=UIMAUpdateRequestProcessorTest -Dtests.method=testMultiplierProcessing -Dtests.seed=1AE011E8E88BC1C9 -Dtests.slow=true -Dtests.locale=lv -Dtests.timezone=Australia/Canberra -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.01s J0 | UIMAUpdateRequestProcessorTest.testMultiplierProcessing <<<
   [junit4]    > Throwable #1: org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
   [junit4]    >   Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([1AE011E8E88BC1C9:21E1DB45EFDFED3D]:0)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
   [junit4]    > 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing(UIMAUpdateRequestProcessorTest.java:83)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testMultiplierProcessing(UIMAUpdateRequestProcessorTest.java:177)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=UIMAUpdateRequestProcessorTest -Dtests.method=testErrorHandling -Dtests.seed=1AE011E8E88BC1C9 -Dtests.slow=true -Dtests.locale=lv -Dtests.timezone=Australia/Canberra -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.02s J0 | UIMAUpdateRequestProcessorTest.testErrorHandling <<<
   [junit4]    > Throwable #1: org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
   [junit4]    >   Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([1AE011E8E88BC1C9:2F146FD6BE5D401D]:0)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
   [junit4]    > 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testErrorHandling(UIMAUpdateRequestProcessorTest.java:137)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/contrib/solr-uima/test/J0/temp/solr.uima.processor.UIMAUpdateRequestProcessorTest_1AE011E8E88BC1C9-001
   [junit4]   2> NOTE: test params are: codec=Lucene70, sim=RandomSimilarity(queryNorm=false): {}, locale=lv, timezone=Australia/Canberra
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=3,threads=1,free=149264728,total=494927872
   [junit4]   2> NOTE: All tests run in this JVM: [UIMAUpdateRequestProcessorTest]
   [junit4] Completed [2/2 (1!)] on J0 in 13.95s, 6 tests, 4 errors <<< FAILURES!

[...truncated 51251 lines...]


[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 974 - Still Unstable!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/974/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseParallelGC

5 tests failed.
FAILED:  org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.testDelegationTokenCancelFail

Error Message:
expected:<200> but was:<404>

Stack Trace:
java.lang.AssertionError: expected:<200> but was:<404>
	at __randomizedtesting.SeedInfo.seed([72A55D89A07D3391:1A1A68A370E7217D]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.junit.Assert.assertEquals(Assert.java:456)
	at org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.cancelDelegationToken(TestSolrCloudWithDelegationTokens.java:140)
	at org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.testDelegationTokenCancelFail(TestSolrCloudWithDelegationTokens.java:304)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testMultiplierProcessing

Error Message:
unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes.    Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>

Stack Trace:
org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
  Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
	at __randomizedtesting.SeedInfo.seed([AEBDCAD9E158122B:95BC0074E60C3EDF]:0)
	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing(UIMAUpdateRequestProcessorTest.java:83)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testMultiplierProcessing(UIMAUpdateRequestProcessorTest.java:177)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testErrorHandling

Error Message:
unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes.    Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>

Stack Trace:
org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
  Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
	at __randomizedtesting.SeedInfo.seed([AEBDCAD9E158122B:9B49B4E7B78E93FF]:0)
	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testErrorHandling(UIMAUpdateRequestProcessorTest.java:137)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testTwoUpdates

Error Message:
unknown UpdateRequestProcessorChain: <add><doc><field name="id">1</field><field name="text">The Apache Software Foundation is happy to announce BarCampApache Sydney, Australia, the first ASF-backed event in the Southern Hemisphere!</field></doc></add>

Stack Trace:
org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">1</field><field name="text">The Apache Software Foundation is happy to announce BarCampApache Sydney, Australia, the first ASF-backed event in the Southern Hemisphere!</field></doc></add>
	at __randomizedtesting.SeedInfo.seed([AEBDCAD9E158122B:A3F79D86F9777085]:0)
	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testTwoUpdates(UIMAUpdateRequestProcessorTest.java:101)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing

Error Message:
unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes.    Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>

Stack Trace:
org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
  Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
	at __randomizedtesting.SeedInfo.seed([AEBDCAD9E158122B:BD33CEABA6AB9381]:0)
	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing(UIMAUpdateRequestProcessorTest.java:83)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 12162 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestSolrCloudWithDelegationTokens
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithDelegationTokens_72A55D89A07D3391-001/init-core-data-001
   [junit4]   2> 2640727 INFO  (SUITE-TestSolrCloudWithDelegationTokens-seed#[72A55D89A07D3391]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 2640730 INFO  (SUITE-TestSolrCloudWithDelegationTokens-seed#[72A55D89A07D3391]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithDelegationTokens_72A55D89A07D3391-001/tempDir-001
   [junit4]   2> 2640731 INFO  (SUITE-TestSolrCloudWithDelegationTokens-seed#[72A55D89A07D3391]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2640731 INFO  (Thread-8739) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2640731 INFO  (Thread-8739) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2640831 INFO  (SUITE-TestSolrCloudWithDelegationTokens-seed#[72A55D89A07D3391]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:40500
   [junit4]   2> 2640843 INFO  (jetty-launcher-3563-thread-2) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2640844 INFO  (jetty-launcher-3563-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@616e43b5{/solr,null,AVAILABLE}
   [junit4]   2> 2640844 INFO  (jetty-launcher-3563-thread-1) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2640845 INFO  (jetty-launcher-3563-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@67502081{SSL,[ssl, http/1.1]}{127.0.0.1:58463}
   [junit4]   2> 2640846 INFO  (jetty-launcher-3563-thread-2) [    ] o.e.j.s.Server Started @2644619ms
   [junit4]   2> 2640846 INFO  (jetty-launcher-3563-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=58463}
   [junit4]   2> 2640846 ERROR (jetty-launcher-3563-thread-2) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2640846 INFO  (jetty-launcher-3563-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 2640846 INFO  (jetty-launcher-3563-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2640846 INFO  (jetty-launcher-3563-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2640846 INFO  (jetty-launcher-3563-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2016-11-27T13:29:39.298Z
   [junit4]   2> 2640849 INFO  (jetty-launcher-3563-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@45ce16ef{/solr,null,AVAILABLE}
   [junit4]   2> 2640851 INFO  (jetty-launcher-3563-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@20483c51{SSL,[ssl, http/1.1]}{127.0.0.1:61146}
   [junit4]   2> 2640851 INFO  (jetty-launcher-3563-thread-1) [    ] o.e.j.s.Server Started @2644624ms
   [junit4]   2> 2640851 INFO  (jetty-launcher-3563-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=61146}
   [junit4]   2> 2640851 ERROR (jetty-launcher-3563-thread-1) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2640851 INFO  (jetty-launcher-3563-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 2640851 INFO  (jetty-launcher-3563-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2640851 INFO  (jetty-launcher-3563-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2640851 INFO  (jetty-launcher-3563-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2016-11-27T13:29:39.303Z
   [junit4]   2> 2640855 INFO  (jetty-launcher-3563-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2640855 INFO  (jetty-launcher-3563-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2640867 INFO  (jetty-launcher-3563-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40500/solr
   [junit4]   2> 2640870 INFO  (jetty-launcher-3563-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40500/solr
   [junit4]   2> 2640906 INFO  (jetty-launcher-3563-thread-2) [n:127.0.0.1:58463_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:58463_solr
   [junit4]   2> 2640908 INFO  (jetty-launcher-3563-thread-1) [n:127.0.0.1:61146_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:61146_solr
   [junit4]   2> 2640908 INFO  (jetty-launcher-3563-thread-2) [n:127.0.0.1:58463_solr    ] o.a.s.c.Overseer Overseer (id=97009885458268165-127.0.0.1:58463_solr-n_0000000000) starting
   [junit4]   2> 2640919 INFO  (jetty-launcher-3563-thread-2) [n:127.0.0.1:58463_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58463_solr
   [junit4]   2> 2640927 INFO  (zkCallback-3573-thread-1-processing-n:127.0.0.1:58463_solr) [n:127.0.0.1:58463_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2640940 INFO  (zkCallback-3574-thread-1-processing-n:127.0.0.1:61146_solr) [n:127.0.0.1:61146_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2640969 INFO  (jetty-launcher-3563-thread-1) [n:127.0.0.1:61146_solr    ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HttpParamDelegationTokenPlugin
   [junit4]   2> 2640971 INFO  (jetty-launcher-3563-thread-2) [n:127.0.0.1:58463_solr    ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HttpParamDelegationTokenPlugin
   [junit4]   2> 2640973 INFO  (zkCallback-3574-thread-1-processing-n:127.0.0.1:61146_solr) [n:127.0.0.1:61146_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2640974 INFO  (OverseerStateUpdate-97009885458268165-127.0.0.1:58463_solr-n_0000000000) [n:127.0.0.1:58463_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2640983 INFO  (jetty-launcher-3563-thread-1) [n:127.0.0.1:61146_solr    ] o.a.s.s.KerberosPlugin Params: {token.valid=30, delegation-token.token-kind=solr-dt, cookie.domain=127.0.0.1, cookie.path=/, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, token.validity=36000, signer.secret.provider.zookeeper.path=/token, type=kerberos, zk-dt-secret-manager.enable=true, kerberos.name.rules=DEFAULT, signer.secret.provider=zookeeper}
   [junit4]   2> 2640983 INFO  (jetty-launcher-3563-thread-2) [n:127.0.0.1:58463_solr    ] o.a.s.s.KerberosPlugin Params: {token.valid=30, delegation-token.token-kind=solr-dt, cookie.domain=127.0.0.1, cookie.path=/, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, token.validity=36000, signer.secret.provider.zookeeper.path=/token, type=kerberos, zk-dt-secret-manager.enable=true, kerberos.name.rules=DEFAULT, signer.secret.provider=zookeeper}
   [junit4]   2> 2641081 INFO  (jetty-launcher-3563-thread-2) [n:127.0.0.1:58463_solr    ] o.a.c.f.i.CuratorFrameworkImpl Starting
   [junit4]   2> 2641081 INFO  (jetty-launcher-3563-thread-1) [n:127.0.0.1:61146_solr    ] o.a.c.f.i.CuratorFrameworkImpl Starting
   [junit4]   2> 2641106 INFO  (jetty-launcher-3563-thread-1-EventThread) [n:127.0.0.1:61146_solr    ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
   [junit4]   2> 2641111 INFO  (jetty-launcher-3563-thread-2-EventThread) [n:127.0.0.1:58463_solr    ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
   [junit4]   2> 2641232 INFO  (jetty-launcher-3563-thread-1) [n:127.0.0.1:61146_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithDelegationTokens_72A55D89A07D3391-001/tempDir-001/node1/.
   [junit4]   2> 2641236 INFO  (jetty-launcher-3563-thread-2) [n:127.0.0.1:58463_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithDelegationTokens_72A55D89A07D3391-001/tempDir-001/node2/.
   [junit4]   2> 2641278 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenSolrClient-seed#[72A55D89A07D3391]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testDelegationTokenSolrClient
   [junit4]   2> 2641366 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenSolrClient-seed#[72A55D89A07D3391]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=jue, 01-ene-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: jue, 01-ene-1970 00:00:00 GMT
   [junit4]   2> 2641384 INFO  (qtp811128200-26301) [n:127.0.0.1:61146_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&delegation=GgADYmFyA2JhcgCKAVil-hS3igFYygaYtwEDFO1jfT1sMu94p_2Di-cy18whOc4bB3NvbHItZHQA&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2641384 INFO  (qtp811128200-26301) [n:127.0.0.1:61146_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&delegation=GgADYmFyA2JhcgCKAVil-hS3igFYygaYtwEDFO1jfT1sMu94p_2Di-cy18whOc4bB3NvbHItZHQA&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2641385 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenSolrClient-seed#[72A55D89A07D3391]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testDelegationTokenSolrClient
   [junit4]   2> 2641413 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationOtherOp-seed#[72A55D89A07D3391]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testDelegationOtherOp
   [junit4]   2> 2641418 INFO  (qtp811128200-26299) [n:127.0.0.1:61146_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params op=someSolrOperation&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2641419 INFO  (qtp811128200-26299) [n:127.0.0.1:61146_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={op=someSolrOperation&action=LIST&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2641419 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationOtherOp-seed#[72A55D89A07D3391]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=bar&p=bar&t=test&e=1480289379870&s=DQBULU+8EN2jmLTnwHPHoHjCYnM="; Path=/; Domain=127.0.0.1; Expires=dom, 27-nov-2016 23:29:39 GMT; Secure; HttpOnly". Invalid 'expires' attribute: dom, 27-nov-2016 23:29:39 GMT
   [junit4]   2> 2641419 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationOtherOp-seed#[72A55D89A07D3391]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testDelegationOtherOp
   [junit4]   2> 2641456 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenewFail-seed#[72A55D89A07D3391]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testDelegationTokenRenewFail
   [junit4]   2> 2641478 WARN  (qtp811128200-26301) [n:127.0.0.1:61146_solr    ] o.a.h.s.a.s.AuthenticationFilter Authentication exception: org.apache.hadoop.security.AccessControlException: foo tries to renew a token with renewer bar
   [junit4]   2> 2641479 WARN  (qtp811128200-26301) [n:127.0.0.1:61146_solr    ] o.a.s.s.KerberosPlugin Could not find org.apache.solr.security.kerberosplugin.requestcontinues
   [junit4]   2> 2641480 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenewFail-seed#[72A55D89A07D3391]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=jue, 01-ene-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: jue, 01-ene-1970 00:00:00 GMT
   [junit4]   2> 2641500 WARN  (qtp2036671657-26289) [    ] o.a.h.s.a.s.AuthenticationFilter Authentication exception: org.apache.hadoop.security.AccessControlException: foo tries to renew a token with renewer bar
   [junit4]   2> 2641501 WARN  (qtp2036671657-26289) [    ] o.a.s.s.KerberosPlugin Could not find org.apache.solr.security.kerberosplugin.requestcontinues
   [junit4]   2> 2641501 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenewFail-seed#[72A55D89A07D3391]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=jue, 01-ene-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: jue, 01-ene-1970 00:00:00 GMT
   [junit4]   2> 2641510 WARN  (qtp811128200-26302) [    ] o.a.h.s.a.s.AuthenticationFilter Authentication exception: org.apache.hadoop.security.AccessControlException: notRenewUser tries to renew a token with renewer renewUser
   [junit4]   2> 2641511 WARN  (qtp811128200-26302) [    ] o.a.s.s.KerberosPlugin Could not find org.apache.solr.security.kerberosplugin.requestcontinues
   [junit4]   2> 2641512 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenewFail-seed#[72A55D89A07D3391]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=jue, 01-ene-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: jue, 01-ene-1970 00:00:00 GMT
   [junit4]   2> 2641514 WARN  (qtp2036671657-26288) [    ] o.a.h.s.a.s.AuthenticationFilter Authentication exception: org.apache.hadoop.security.AccessControlException: notRenewUser tries to renew a token with renewer renewUser
   [junit4]   2> 2641514 WARN  (qtp2036671657-26288) [    ] o.a.s.s.KerberosPlugin Could not find org.apache.solr.security.kerberosplugin.requestcontinues
   [junit4]   2> 2641515 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenewFail-seed#[72A55D89A07D3391]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=jue, 01-ene-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: jue, 01-ene-1970 00:00:00 GMT
   [junit4]   2> 2641515 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenewFail-seed#[72A55D89A07D3391]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testDelegationTokenRenewFail
   [junit4]   2> 2641548 INFO  (TEST-TestSolrCloudWithDelegationTokens.testZNodePaths-seed#[72A55D89A07D3391]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testZNodePaths
   [junit4]   2> 2641567 INFO  (TEST-TestSolrCloudWithDelegationTokens.testZNodePaths-seed#[72A55D89A07D3391]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testZNodePaths
   [junit4]   2> 2641594 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenVerify-seed#[72A55D89A07D3391]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testDelegationTokenVerify
   [junit4]   2> 2641614 INFO  (qtp811128200-26298) [n:127.0.0.1:61146_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&delegation=GgADYmFyA2JhcgCKAVil-hXPigFYygaZzwUDFGeMZW_9WUXZ5lPj2_AY77O9EMKDB3NvbHItZHQA&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2641614 INFO  (qtp811128200-26298) [n:127.0.0.1:61146_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&delegation=GgADYmFyA2JhcgCKAVil-hXPigFYygaZzwUDFGeMZW_9WUXZ5lPj2_AY77O9EMKDB3NvbHItZHQA&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2641620 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenVerify-seed#[72A55D89A07D3391]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2641622 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenVerify-seed#[72A55D89A07D3391]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40500/solr ready
   [junit4]   2> 2641626 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenVerify-seed#[72A55D89A07D3391]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=jue, 01-ene-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: jue, 01-ene-1970 00:00:00 GMT
   [junit4]   2> 2641637 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenVerify-seed#[72A55D89A07D3391]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2641638 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenVerify-seed#[72A55D89A07D3391]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40500/solr ready
   [junit4]   2> 2641657 INFO  (qtp811128200-26301) [n:127.0.0.1:61146_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&delegation=GgADYmFyA2JhcgCKAVil-hXPigFYygaZzwUDFGeMZW_9WUXZ5lPj2_AY77O9EMKDB3NvbHItZHQA&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2641657 INFO  (qtp811128200-26301) [n:127.0.0.1:61146_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&delegation=GgADYmFyA2JhcgCKAVil-hXPigFYygaZzwUDFGeMZW_9WUXZ5lPj2_AY77O9EMKDB3NvbHItZHQA&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2641675 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenVerify-seed#[72A55D89A07D3391]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=jue, 01-ene-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: jue, 01-ene-1970 00:00:00 GMT
   [junit4]   2> 2641675 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenVerify-seed#[72A55D89A07D3391]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testDelegationTokenVerify
   [junit4]   2> 2641702 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancel-seed#[72A55D89A07D3391]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testDelegationTokenCancel
   [junit4]   2> 2641751 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancel-seed#[72A55D89A07D3391]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2641752 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancel-seed#[72A55D89A07D3391]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40500/solr ready
   [junit4]   2> 2641757 ERROR (qtp811128200-26299) [n:127.0.0.1:61146_solr    ] o.a.h.s.t.d.ZKDelegationTokenSecretManager No node in path [/ZKDTSMTokensRoot/DT_6]
   [junit4]   2> 2641768 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancel-seed#[72A55D89A07D3391]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2641770 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancel-seed#[72A55D89A07D3391]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40500/solr ready
   [junit4]   2> 2641774 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancel-seed#[72A55D89A07D3391]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=jue, 01-ene-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: jue, 01-ene-1970 00:00:00 GMT
   [junit4]   2> 2641780 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancel-seed#[72A55D89A07D3391]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2641781 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancel-seed#[72A55D89A07D3391]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40500/solr ready
   [junit4]   2> 2641785 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancel-seed#[72A55D89A07D3391]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=jue, 01-ene-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: jue, 01-ene-1970 00:00:00 GMT
   [junit4]   2> 2641800 ERROR (qtp2036671657-26288) [    ] o.a.h.s.t.d.ZKDelegationTokenSecretManager No node in path [/ZKDTSMTokensRoot/DT_7]
   [junit4]   2> 2641807 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancel-seed#[72A55D89A07D3391]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=jue, 01-ene-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: jue, 01-ene-1970 00:00:00 GMT
   [junit4]   2> 2641816 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancel-seed#[72A55D89A07D3391]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2641818 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancel-seed#[72A55D89A07D3391]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40500/solr ready
   [junit4]   2> 2641821 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancel-seed#[72A55D89A07D3391]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=jue, 01-ene-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: jue, 01-ene-1970 00:00:00 GMT
   [junit4]   2> 2641822 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancel-seed#[72A55D89A07D3391]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testDelegationTokenCancel
   [junit4]   2> 2641863 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancelFail-seed#[72A55D89A07D3391]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testDelegationTokenCancelFail
   [junit4]   2> 2641874 ERROR (qtp2036671657-26290) [    ] o.a.h.s.t.d.ZKDelegationTokenSecretManager No node in path [/ZKDTSMTokensRoot/DT_8]
   [junit4]   2> 2641876 ERROR (qtp811128200-26299) [n:127.0.0.1:61146_solr    ] o.a.h.s.t.d.ZKDelegationTokenSecretManager No node in path [/ZKDTSMTokensRoot/DT_8]
   [junit4]   2> 2641886 ERROR (qtp2036671657-26303) [    ] o.a.h.s.t.d.ZKDelegationTokenSecretManager No node in path [/ZKDTSMTokensRoot/DT_9]
   [junit4]   2> 2641896 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancelFail-seed#[72A55D89A07D3391]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testDelegationTokenCancelFail
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestSolrCloudWithDelegationTokens -Dtests.method=testDelegationTokenCancelFail -Dtests.seed=72A55D89A07D3391 -Dtests.slow=true -Dtests.locale=es-PE -Dtests.timezone=MET -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] FAILURE 0.08s J0 | TestSolrCloudWithDelegationTokens.testDelegationTokenCancelFail <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: expected:<200> but was:<404>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([72A55D89A07D3391:1A1A68A370E7217D]:0)
   [junit4]    > 	at org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.cancelDelegationToken(TestSolrCloudWithDelegationTokens.java:140)
   [junit4]    > 	at org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.testDelegationTokenCancelFail(TestSolrCloudWithDelegationTokens.java:304)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2641940 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[72A55D89A07D3391]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testDelegationTokenRenew
   [junit4]   2> 2641964 INFO  (qtp811128200-26299) [n:127.0.0.1:61146_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&delegation=GgADYmFyA2JhcgCKAVil-hcpigFYygabKQoDFAZB_q_nGgsLWRlJraL-uNXXQHdkB3NvbHItZHQA&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2641964 INFO  (qtp811128200-26299) [n:127.0.0.1:61146_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&delegation=GgADYmFyA2JhcgCKAVil-hcpigFYygabKQoDFAZB_q_nGgsLWRlJraL-uNXXQHdkB3NvbHItZHQA&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2641969 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[72A55D89A07D3391]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=jue, 01-ene-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: jue, 01-ene-1970 00:00:00 GMT
   [junit4]   2> 2641974 INFO  (qtp2036671657-26287) [n:127.0.0.1:58463_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&delegation=GgADYmFyA2JhcgCKAVil-hcpigFYygabKQoDFAZB_q_nGgsLWRlJraL-uNXXQHdkB3NvbHItZHQA&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2641975 INFO  (qtp2036671657-26287) [n:127.0.0.1:58463_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&delegation=GgADYmFyA2JhcgCKAVil-hcpigFYygabKQoDFAZB_q_nGgsLWRlJraL-uNXXQHdkB3NvbHItZHQA&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2641981 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[72A55D89A07D3391]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2641983 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[72A55D89A07D3391]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40500/solr ready
   [junit4]   2> 2641987 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[72A55D89A07D3391]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=jue, 01-ene-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: jue, 01-ene-1970 00:00:00 GMT
   [junit4]   2> 2642010 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[72A55D89A07D3391]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2642011 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[72A55D89A07D3391]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40500/solr ready
   [junit4]   2> 2642016 INFO  (qtp811128200-26299) [n:127.0.0.1:61146_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&delegation=GgADYmFyA2JhcgCKAVil-hdZigFYygabWQsDFKnDjIJymJ5_nskjCYZSLGqcYRxZB3NvbHItZHQA&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2642016 INFO  (qtp811128200-26299) [n:127.0.0.1:61146_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&delegation=GgADYmFyA2JhcgCKAVil-hdZigFYygabWQsDFKnDjIJymJ5_nskjCYZSLGqcYRxZB3NvbHItZHQA&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2642022 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[72A55D89A07D3391]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=jue, 01-ene-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: jue, 01-ene-1970 00:00:00 GMT
   [junit4]   2> 2642055 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[72A55D89A07D3391]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2642056 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[72A55D89A07D3391]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40500/solr ready
   [junit4]   2> 2642061 INFO  (qtp811128200-26299) [n:127.0.0.1:61146_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&delegation=GgADYmFyA2JhcgCKAVil-hdZigFYygabWQsDFKnDjIJymJ5_nskjCYZSLGqcYRxZB3NvbHItZHQA&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2642061 INFO  (qtp811128200-26299) [n:127.0.0.1:61146_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&delegation=GgADYmFyA2JhcgCKAVil-hdZigFYygabWQsDFKnDjIJymJ5_nskjCYZSLGqcYRxZB3NvbHItZHQA&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2642069 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[72A55D89A07D3391]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=jue, 01-ene-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: jue, 01-ene-1970 00:00:00 GMT
   [junit4]   2> 2642094 INFO  (qtp811128200-26293) [n:127.0.0.1:61146_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&delegation=GgADYmFyA2JhcgCKAVil-heqigFYygabqgwDFCyNqL-ecTV_9KEVGj1lRDtaxaP1B3NvbHItZHQA&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2642094 INFO  (qtp811128200-26293) [n:127.0.0.1:61146_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&delegation=GgADYmFyA2JhcgCKAVil-heqigFYygabqgwDFCyNqL-ecTV_9KEVGj1lRDtaxaP1B3NvbHItZHQA&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2642100 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[72A55D89A07D3391]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2642102 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[72A55D89A07D3391]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40500/solr ready
   [junit4]   2> 2642106 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[72A55D89A07D3391]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=jue, 01-ene-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: jue, 01-ene-1970 00:00:00 GMT
   [junit4]   2> 2642114 INFO  (qtp2036671657-26304) [n:127.0.0.1:58463_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&delegation=GgADYmFyA2JhcgCKAVil-heqigFYygabqgwDFCyNqL-ecTV_9KEVGj1lRDtaxaP1B3NvbHItZHQA&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2642114 INFO  (qtp2036671657-26304) [n:127.0.0.1:58463_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&delegation=GgADYmFyA2JhcgCKAVil-heqigFYygabqgwDFCyNqL-ecTV_9KEVGj1lRDtaxaP1B3NvbHItZHQA&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2642119 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[72A55D89A07D3391]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2642120 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[72A55D89A07D3391]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40500/solr ready
   [junit4]   2> 2642125 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[72A55D89A07D3391]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=jue, 01-ene-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: jue, 01-ene-1970 00:00:00 GMT
   [junit4]   2> 2642143 INFO  (qtp811128200-26292) [n:127.0.0.1:61146_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&delegation=GgADYmFyA2JhcgCKAVil-hfiigFYygab4g0DFGa5i9UK3R17Flq0IWSN0jtAkVzVB3NvbHItZHQA&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2642143 INFO  (qtp811128200-26292) [n:127.0.0.1:61146_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&delegation=GgADYmFyA2JhcgCKAVil-hfiigFYygab4g0DFGa5i9UK3R17Flq0IWSN0jtAkVzVB3NvbHItZHQA&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2642149 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[72A55D89A07D3391]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=jue, 01-ene-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: jue, 01-ene-1970 00:00:00 GMT
   [junit4]   2> 2642155 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[72A55D89A07D3391]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2642156 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[72A55D89A07D3391]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40500/solr ready
   [junit4]   2> 2642161 INFO  (qtp811128200-26292) [n:127.0.0.1:61146_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&delegation=GgADYmFyA2JhcgCKAVil-hfiigFYygab4g0DFGa5i9UK3R17Flq0IWSN0jtAkVzVB3NvbHItZHQA&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2642161 INFO  (qtp811128200-26292) [n:127.0.0.1:61146_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&delegation=GgADYmFyA2JhcgCKAVil-hfiigFYygab4g0DFGa5i9UK3R17Flq0IWSN0jtAkVzVB3NvbHItZHQA&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2642167 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[72A55D89A07D3391]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2642169 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[72A55D89A07D3391]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40500/solr ready
   [junit4]   2> 2642173 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[72A55D89A07D3391]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=jue, 01-ene-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: jue, 01-ene-1970 00:00:00 GMT
   [junit4]   2> 2642175 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[72A55D89A07D3391]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testDelegationTokenRenew
   [junit4]   2> 2642176 INFO  (jetty-launcher-3563-thread-1) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@67502081{SSL,[ssl, http/1.1]}{127.0.0.1:0}
   [junit4]   2> 2642176 INFO  (jetty-launcher-3563-thread-2) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@20483c51{SSL,[ssl, http/1.1]}{127.0.0.1:0}
   [junit4]   2> 2642177 INFO  (jetty-launcher-3563-thread-1) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1524632922
   [junit4]   2> 2642178 INFO  (jetty-launcher-3563-thread-2) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=309121121
   [junit4]   2> 2642181 INFO  (jetty-launcher-3563-thread-1) [    ] o.a.s.c.Overseer Overseer (id=97009885458268165-127.0.0.1:58463_solr-n_0000000000) closing
   [junit4]   2> 2642183 INFO  (OverseerStateUpdate-97009885458268165-127.0.0.1:58463_solr-n_0000000000) [n:127.0.0.1:58463_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:58463_solr
   [junit4]   2> 2642187 INFO  (zkCallback-3574-thread-1-processing-n:127.0.0.1:61146_solr) [n:127.0.0.1:61146_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:61146_solr
   [junit4]   2> 2642188 INFO  (zkCallback-3574-thread-1-processing-n:127.0.0.1:61146_solr) [n:127.0.0.1:61146_solr    ] o.a.s.c.Overseer Overseer (id=97009885458268166-127.0.0.1:61146_solr-n_0000000001) starting
   [junit4]   2> 2642203 INFO  (OverseerStateUpdate-97009885458268166-127.0.0.1:61146_solr-n_0000000001) [n:127.0.0.1:61146_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 2642209 INFO  (jetty-launcher-3563-thread-2) [    ] o.a.s.c.Overseer Overseer (id=97009885458268166-127.0.0.1:61146_solr-n_0000000001) closing
   [junit4]   2> 2642211 INFO  (OverseerStateUpdate-97009885458268166-127.0.0.1:61146_solr-n_0000000001) [n:127.0.0.1:61146_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:61146_solr
   [junit4]   2> 2642214 ERROR (Thread[Thread-8759,5,TGRP-TestSolrCloudWithDelegationTokens]) [n:127.0.0.1:61146_solr    ] o.a.h.s.t.d.AbstractDelegationTokenSecretManager ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 2642226 INFO  (jetty-launcher-3563-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@45ce16ef{/solr,null,UNAVAILABLE}
   [junit4]   2> 2643685 WARN  (zkCallback-3573-thread-1-processing-n:127.0.0.1:58463_solr) [n:127.0.0.1:58463_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 2643685 ERROR (Thread[Thread-8762,5,TGRP-TestSolrCloudWithDelegationTokens]) [n:127.0.0.1:58463_solr    ] o.a.h.s.t.d.AbstractDelegationTokenSecretManager ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 2643690 INFO  (jetty-launcher-3563-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@616e43b5{/solr,null,UNAVAILABLE}
   [junit4]   2> 2643693 INFO  (SUITE-TestSolrCloudWithDelegationTokens-seed#[72A55D89A07D3391]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40500 40500
   [junit4]   2> 2643720 INFO  (Thread-8739) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40500 40500
   [junit4]   2> 2643721 WARN  (Thread-8739) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	15	/solr/aliases.json
   [junit4]   2> 	2	/solr/security.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	15	/solr/clusterprops.json
   [junit4]   2> 	15	/solr/clusterstate.json
   [junit4]   2> 	4	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMKeyIdRoot
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_6
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_1
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_1
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_3
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_3
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_2
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_2
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_5
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_4
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_4
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_13
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_12
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMSeqNumRoot
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_11
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_10
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	15	/solr/live_nodes
   [junit4]   2> 	15	/solr/collections
   [junit4]   2> 	4	/solr/overseer/queue
   [junit4]   2> 	3	/solr/overseer/queue-work
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot
   [junit4]   2> 	2	/solr/overseer/collection-queue-work
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot
   [junit4]   2> 
   [junit4]   2> 2643722 INFO  (SUITE-TestSolrCloudWithDelegationTokens-seed#[72A55D89A07D3391]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithDelegationTokens_72A55D89A07D3391-001
   [junit4]   2> Nov 27, 2016 1:29:42 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=FastDecompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=2, maxDocsPerChunk=430, blockSize=805), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=2, blockSize=805)), sim=RandomSimilarity(queryNorm=true): {}, locale=es-PE, timezone=MET
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=3,threads=1,free=95404240,total=526385152
   [junit4]   2> NOTE: All tests run in this JVM: [MigrateRouteKeyTest, TestBM25SimilarityFactory, TestCursorMarkWithoutUniqueKey, CoreAdminRequestStatusTest, TestSweetSpotSimilarityFactory, TestManagedResource, NumericFieldsTest, TestSimpleQParserPlugin, TestSort, SpatialRPTFieldTypeTest, SampleTest, SolrTestCaseJ4Test, TestSolrCloudSnapshots, CdcrReplicationDistributedZkTest, DeleteLastCustomShardedReplicaTest, TestCorePropertiesReload, ZkCLITest, DocumentAnalysisRequestHandlerTest, TestReplicationHandler, TestSchemaResource, TestUpdate, TestPostingsSolrHighlighter, JSONWriterTest, CdcrRequestHandlerTest, TestConfigOverlay, TestSolrDynamicMBean, TestClassicSimilarityFactory, SearchHandlerTest, TestPKIAuthenticationPlugin, TestDynamicLoading, TestXIncludeConfig, DisMaxRequestHandlerTest, SyncSliceTest, TestGroupingSearch, TestReplicationHandlerBackup, BaseCdcrDistributedZkTest, TestFunctionQuery, TestReplicaProperties, CustomCollectionTest, ReplaceNodeTest, PluginInfoTest, TestWordDelimiterFilterFactory, TestExclusionRuleCollectionAccess, AlternateDirectoryTest, ExplicitHLLTest, HdfsRecoverLeaseTest, DeleteReplicaTest, DistributedQueryElevationComponentTest, SystemInfoHandlerTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestRequestStatusCollectionAPI, DistributedQueueTest, TestCoreContainer, TestCSVLoader, TestLegacyFieldCache, CloudExitableDirectoryReaderTest, DistributedDebugComponentTest, HighlighterTest, EnumFieldTest, UUIDUpdateProcessorFallbackTest, TestPerFieldSimilarity, SolrXmlInZkTest, OverseerCollectionConfigSetProcessorTest, TestPhraseSuggestions, SimplePostToolTest, QueryEqualityTest, TestSurroundQueryParser, PeerSyncWithIndexFingerprintCachingTest, TestCloudDeleteByQuery, TestBinaryField, TestFieldResource, MBeansHandlerTest, DistribDocExpirationUpdateProcessorTest, TestFieldCacheSort, ShardRoutingTest, TestRestoreCore, TestJettySolrRunner, TestTrieFacet, SuggesterFSTTest, TestSolr4Spatial2, RecoveryAfterSoftCommitTest, ZkStateReaderTest, DistributedSpellCheckComponentTest, HdfsChaosMonkeyNothingIsSafeTest, TestStressLucene, TestCollectionAPI, RequestLoggingTest, SolrRequestParserTest, SparseHLLTest, RankQueryTest, TestInfoStreamLogging, LeaderFailoverAfterPartitionTest, TestLFUCache, AssignTest, DistributedQueryComponentCustomSortTest, BasicDistributedZkTest, TestTolerantUpdateProcessorRandomCloud, TestHighlightDedupGrouping, TestLazyCores, TestFieldTypeResource, TestCharFilters, HLLSerializationTest, HdfsRestartWhileUpdatingTest, TestSQLHandlerNonCloud, TestSha256AuthenticationProvider, TestRTGBase, TestSubQueryTransformerCrossCore, AnalyticsMergeStrategyTest, LeaderInitiatedRecoveryOnCommitTest, TestUniqueKeyFieldResource, TestOrdValues, TestNamedUpdateProcessors, TestReloadDeadlock, DistributedFacetPivotLongTailTest, TestTolerantSearch, TestConfigSetProperties, OpenExchangeRatesOrgProviderTest, BigEndianAscendingWordSerializerTest, HdfsBasicDistributedZk2Test, PrimitiveFieldTypeTest, AnalysisAfterCoreReloadTest, SpatialHeatmapFacetsTest, TestAnalyzeInfixSuggestions, BJQParserTest, DistributedMLTComponentTest, StandardRequestHandlerTest, SimpleFacetsTest, StatelessScriptUpdateProcessorFactoryTest, UnloadDistributedZkTest, TestScoreJoinQPScore, PrimUtilsTest, PeerSyncTest, TestPseudoReturnFields, TestLeaderInitiatedRecoveryThread, DirectSolrConnectionTest, PathHierarchyTokenizerFactoryTest, ConcurrentDeleteAndCreateCollectionTest, TestCryptoKeys, TestCloudSchemaless, DateFieldTest, TestUseDocValuesAsStored, TestRawTransformer, PingRequestHandlerTest, LeaderElectionTest, TestPushWriter, TestStandardQParsers, ReplicaListTransformerTest, TestFaceting, TestHighFrequencyDictionaryFactory, CheckHdfsIndexTest, SolrCLIZkUtilsTest, TestManagedSynonymFilterFactory, TestMergePolicyConfig, TestDefaultSearchFieldResource, StressHdfsTest, CurrencyFieldXmlFileTest, DeleteStatusTest, ResponseHeaderTest, DocValuesTest, TestCloudPivotFacet, OpenCloseCoreStressTest, OverseerTest, RecoveryZkTest, FullSolrCloudDistribCmdsTest, ClusterStateUpdateTest, ZkSolrClientTest, TestZkChroot, TestRandomDVFaceting, TestDistributedSearch, AutoCommitTest, TestHashPartitioner, TermVectorComponentDistributedTest, ZkControllerTest, TestRealTimeGet, TestStressReorder, TestReload, TestStressVersions, HardAutoCommitTest, DistributedTermsComponentTest, SolrCoreTest, StatsComponentTest, SpellCheckComponentTest, SolrCmdDistributorTest, QueryElevationComponentTest, BasicFunctionalityTest, DirectUpdateHandlerTest, TestBadConfig, SoftAutoCommitTest, ShowFileRequestHandlerTest, CurrencyFieldOpenExchangeTest, SolrIndexSplitterTest, TestCoreDiscovery, SignatureUpdateProcessorFactoryTest, TestExtendedDismaxParser, CoreAdminHandlerTest, SpatialFilterTest, NoCacheHeaderTest, TestAtomicUpdateErrorCases, TestRemoteStreaming, TestSolrDeletionPolicy1, IndexBasedSpellCheckerTest, LukeRequestHandlerTest, TestQueryTypes, TestOmitPositions, XmlUpdateRequestHandlerTest, TestValueSourceCache, RequiredFieldsTest, IndexSchemaRuntimeFieldTest, RegexBoostProcessorTest, TestJmxIntegration, UpdateRequestProcessorFactoryTest, TestCSVResponseWriter, UniqFieldsUpdateProcessorFactoryTest, JsonLoaderTest, HighlighterConfigTest, TestSolrIndexConfig, CopyFieldTest, BadComponentTest, TestStressRecovery, ActionThrottleTest, CdcrReplicationHandlerTest, CdcrVersionReplicationTest, CleanupOldIndexTest, CollectionStateFormat2Test, CollectionTooManyReplicasTest, ConfigSetsAPITest, ConnectionManagerTest, DistribCursorPagingTest, LeaderElectionContextKeyTest, LeaderFailureAfterFreshStartTest, LeaderInitiatedRecoveryOnShardRestartTest, MultiThreadedOCPTest, OutOfBoxZkACLAndCredentialsProvidersTest, OverseerModifyCollectionTest, OverseerTaskQueueTest, RemoteQueryErrorTest, SSLMigrationTest, ShardSplitTest, TestRequestForwarding, TestSSLRandomization, TestShortCircuitedRequests, TestSolrCloudWithDelegationTokens]
   [junit4] Completed [500/658 (1!)] on J0 in 4.03s, 8 tests, 1 failure <<< FAILURES!

[...truncated 11687 lines...]
   [junit4] Suite: org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/contrib/solr-uima/test/J0/temp/solr.uima.processor.UIMAUpdateRequestProcessorTest_AEBDCAD9E158122B-001/init-core-data-001
   [junit4]   2> log4j:WARN No appenders could be found for logger (org.eclipse.jetty.util.log).
   [junit4]   2> log4j:WARN Please initialize the log4j system properly.
   [junit4]   2> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
   [junit4]   2> Nov 27, 2016 12:44:55 PM WhitespaceTokenizer initialize
   [junit4]   2> INFO: "Whitespace tokenizer successfully initialized"
   [junit4]   2> Nov 27, 2016 12:44:59 PM WhitespaceTokenizer initialize
   [junit4]   2> INFO: "Whitespace tokenizer successfully initialized"
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=UIMAUpdateRequestProcessorTest -Dtests.method=testMultiplierProcessing -Dtests.seed=AEBDCAD9E158122B -Dtests.slow=true -Dtests.locale=en-ZA -Dtests.timezone=Etc/GMT+1 -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.01s J0 | UIMAUpdateRequestProcessorTest.testMultiplierProcessing <<<
   [junit4]    > Throwable #1: org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
   [junit4]    >   Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([AEBDCAD9E158122B:95BC0074E60C3EDF]:0)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
   [junit4]    > 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing(UIMAUpdateRequestProcessorTest.java:83)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testMultiplierProcessing(UIMAUpdateRequestProcessorTest.java:177)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=UIMAUpdateRequestProcessorTest -Dtests.method=testErrorHandling -Dtests.seed=AEBDCAD9E158122B -Dtests.slow=true -Dtests.locale=en-ZA -Dtests.timezone=Etc/GMT+1 -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.02s J0 | UIMAUpdateRequestProcessorTest.testErrorHandling <<<
   [junit4]    > Throwable #1: org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
   [junit4]    >   Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([AEBDCAD9E158122B:9B49B4E7B78E93FF]:0)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
   [junit4]    > 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testErrorHandling(UIMAUpdateRequestProcessorTest.java:137)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=UIMAUpdateRequestProcessorTest -Dtests.method=testTwoUpdates -Dtests.seed=AEBDCAD9E158122B -Dtests.slow=true -Dtests.locale=en-ZA -Dtests.timezone=Etc/GMT+1 -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.01s J0 | UIMAUpdateRequestProcessorTest.testTwoUpdates <<<
   [junit4]    > Throwable #1: org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">1</field><field name="text">The Apache Software Foundation is happy to announce BarCampApache Sydney, Australia, the first ASF-backed event in the Southern Hemisphere!</field></doc></add>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([AEBDCAD9E158122B:A3F79D86F9777085]:0)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
   [junit4]    > 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testTwoUpdates(UIMAUpdateRequestProcessorTest.java:101)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=UIMAUpdateRequestProcessorTest -Dtests.method=testProcessing -Dtests.seed=AEBDCAD9E158122B -Dtests.slow=true -Dtests.locale=en-ZA -Dtests.timezone=Etc/GMT+1 -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.01s J0 | UIMAUpdateRequestProcessorTest.testProcessing <<<
   [junit4]    > Throwable #1: org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
   [junit4]    >   Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([AEBDCAD9E158122B:BD33CEABA6AB9381]:0)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
   [junit4]    > 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing(UIMAUpdateRequestProcessorTest.java:83)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/contrib/solr-uima/test/J0/temp/solr.uima.processor.UIMAUpdateRequestProcessorTest_AEBDCAD9E158122B-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=269, maxMBSortInHeap=5.856874229042178, sim=RandomSimilarity(queryNorm=false): {}, locale=en-ZA, timezone=Etc/GMT+1
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=3,threads=1,free=168251216,total=438304768
   [junit4]   2> NOTE: All tests run in this JVM: [UIMAUpdateRequestProcessorTest]
   [junit4] Completed [2/2 (1!)] on J0 in 13.96s, 6 tests, 4 errors <<< FAILURES!

[...truncated 51250 lines...]


[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 973 - Still Unstable!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/973/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseG1GC

7 tests failed.
FAILED:  org.apache.solr.cloud.CdcrBootstrapTest.testBootstrapWithSourceCluster

Error Message:
Document mismatch on target after sync expected:<20000> but was:<10000>

Stack Trace:
java.lang.AssertionError: Document mismatch on target after sync expected:<20000> but was:<10000>
	at __randomizedtesting.SeedInfo.seed([8080C71C649636E9:59D696D867F225A3]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.CdcrBootstrapTest.testBootstrapWithSourceCluster(CdcrBootstrapTest.java:232)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.DocValuesNotIndexedTest.testGroupingSorting

Error Message:
Documents in wrong order for field: boolGSF expected:<[2]> but was:<[1]>

Stack Trace:
org.junit.ComparisonFailure: Documents in wrong order for field: boolGSF expected:<[2]> but was:<[1]>
	at __randomizedtesting.SeedInfo.seed([8080C71C649636E9:9EB8CF14183D8C69]:0)
	at org.junit.Assert.assertEquals(Assert.java:125)
	at org.apache.solr.cloud.DocValuesNotIndexedTest.checkSortOrder(DocValuesNotIndexedTest.java:247)
	at org.apache.solr.cloud.DocValuesNotIndexedTest.testGroupingSorting(DocValuesNotIndexedTest.java:227)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.HttpPartitionTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=19483, name=SocketProxy-Response-36324:48361, state=RUNNABLE, group=TGRP-HttpPartitionTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=19483, name=SocketProxy-Response-36324:48361, state=RUNNABLE, group=TGRP-HttpPartitionTest]
	at __randomizedtesting.SeedInfo.seed([8080C71C649636E9:8D4F8C6CA6A5B11]:0)
Caused by: java.lang.RuntimeException: java.net.SocketException: Socket is closed
	at __randomizedtesting.SeedInfo.seed([8080C71C649636E9]:0)
	at org.apache.solr.cloud.SocketProxy$Bridge$Pump.run(SocketProxy.java:347)
Caused by: java.net.SocketException: Socket is closed
	at java.net.Socket.setSoTimeout(Socket.java:1137)
	at org.apache.solr.cloud.SocketProxy$Bridge$Pump.run(SocketProxy.java:344)


FAILED:  org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testErrorHandling

Error Message:
unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes.    Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>

Stack Trace:
org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
  Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
	at __randomizedtesting.SeedInfo.seed([3D0313FFFB63451D:8F76DC1ADB5C4C9]:0)
	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testErrorHandling(UIMAUpdateRequestProcessorTest.java:137)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testMultiplierProcessing

Error Message:
unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes.    Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>

Stack Trace:
org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
  Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
	at __randomizedtesting.SeedInfo.seed([3D0313FFFB63451D:602D952FC3769E9]:0)
	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing(UIMAUpdateRequestProcessorTest.java:83)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testMultiplierProcessing(UIMAUpdateRequestProcessorTest.java:177)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing

Error Message:
unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes.    Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>

Stack Trace:
org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
  Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
	at __randomizedtesting.SeedInfo.seed([3D0313FFFB63451D:2E8D178DBC90C4B7]:0)
	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing(UIMAUpdateRequestProcessorTest.java:83)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testTwoUpdates

Error Message:
unknown UpdateRequestProcessorChain: <add><doc><field name="id">1</field><field name="text">The Apache Software Foundation is happy to announce BarCampApache Sydney, Australia, the first ASF-backed event in the Southern Hemisphere!</field></doc></add>

Stack Trace:
org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">1</field><field name="text">The Apache Software Foundation is happy to announce BarCampApache Sydney, Australia, the first ASF-backed event in the Southern Hemisphere!</field></doc></add>
	at __randomizedtesting.SeedInfo.seed([3D0313FFFB63451D:304944A0E34C27B3]:0)
	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testTwoUpdates(UIMAUpdateRequestProcessorTest.java:101)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11450 lines...]
   [junit4] Suite: org.apache.solr.cloud.DocValuesNotIndexedTest
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.DocValuesNotIndexedTest_8080C71C649636E9-001/init-core-data-001
   [junit4]   2> 1158821 INFO  (SUITE-DocValuesNotIndexedTest-seed#[8080C71C649636E9]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 1158822 INFO  (SUITE-DocValuesNotIndexedTest-seed#[8080C71C649636E9]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.DocValuesNotIndexedTest_8080C71C649636E9-001/tempDir-001
   [junit4]   2> 1158822 INFO  (SUITE-DocValuesNotIndexedTest-seed#[8080C71C649636E9]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1158822 INFO  (Thread-2680) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1158822 INFO  (Thread-2680) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1158922 INFO  (SUITE-DocValuesNotIndexedTest-seed#[8080C71C649636E9]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:58011
   [junit4]   2> 1158931 INFO  (jetty-launcher-1630-thread-2) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 1158932 INFO  (jetty-launcher-1630-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@64e4929c{/solr,null,AVAILABLE}
   [junit4]   2> 1158932 INFO  (jetty-launcher-1630-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@197367e1{HTTP/1.1,[http/1.1]}{127.0.0.1:44610}
   [junit4]   2> 1158932 INFO  (jetty-launcher-1630-thread-2) [    ] o.e.j.s.Server Started @1162829ms
   [junit4]   2> 1158932 INFO  (jetty-launcher-1630-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44610}
   [junit4]   2> 1158932 ERROR (jetty-launcher-1630-thread-2) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1158932 INFO  (jetty-launcher-1630-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solrâ„¢ version 7.0.0
   [junit4]   2> 1158932 INFO  (jetty-launcher-1630-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1158932 INFO  (jetty-launcher-1630-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1158933 INFO  (jetty-launcher-1630-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2016-11-27T02:53:49.329Z
   [junit4]   2> 1158939 INFO  (jetty-launcher-1630-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1158947 INFO  (jetty-launcher-1630-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:58011/solr
   [junit4]   2> 1158982 INFO  (jetty-launcher-1630-thread-2) [n:127.0.0.1:44610_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44610_solr
   [junit4]   2> 1158984 INFO  (jetty-launcher-1630-thread-2) [n:127.0.0.1:44610_solr    ] o.a.s.c.Overseer Overseer (id=97007385262161923-127.0.0.1:44610_solr-n_0000000000) starting
   [junit4]   2> 1158994 INFO  (jetty-launcher-1630-thread-2) [n:127.0.0.1:44610_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44610_solr
   [junit4]   2> 1159023 INFO  (jetty-launcher-1630-thread-1) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 1159025 INFO  (jetty-launcher-1630-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2b909907{/solr,null,AVAILABLE}
   [junit4]   2> 1159025 INFO  (jetty-launcher-1630-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@55c1ac66{HTTP/1.1,[http/1.1]}{127.0.0.1:55150}
   [junit4]   2> 1159025 INFO  (jetty-launcher-1630-thread-1) [    ] o.e.j.s.Server Started @1162922ms
   [junit4]   2> 1159025 INFO  (jetty-launcher-1630-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=55150}
   [junit4]   2> 1159025 ERROR (jetty-launcher-1630-thread-1) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1159025 INFO  (jetty-launcher-1630-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solrâ„¢ version 7.0.0
   [junit4]   2> 1159025 INFO  (jetty-launcher-1630-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1159026 INFO  (jetty-launcher-1630-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1159026 INFO  (jetty-launcher-1630-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2016-11-27T02:53:49.422Z
   [junit4]   2> 1159034 INFO  (zkCallback-1636-thread-1-processing-n:127.0.0.1:44610_solr) [n:127.0.0.1:44610_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1159039 INFO  (jetty-launcher-1630-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1159045 INFO  (jetty-launcher-1630-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:58011/solr
   [junit4]   2> 1159058 INFO  (jetty-launcher-1630-thread-1) [n:127.0.0.1:55150_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1159064 INFO  (jetty-launcher-1630-thread-1) [n:127.0.0.1:55150_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55150_solr
   [junit4]   2> 1159092 INFO  (zkCallback-1642-thread-1-processing-n:127.0.0.1:55150_solr) [n:127.0.0.1:55150_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1159108 INFO  (zkCallback-1636-thread-2-processing-n:127.0.0.1:44610_solr) [n:127.0.0.1:44610_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1159116 INFO  (jetty-launcher-1630-thread-1) [n:127.0.0.1:55150_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.DocValuesNotIndexedTest_8080C71C649636E9-001/tempDir-001/node2/.
   [junit4]   2> 1159189 INFO  (jetty-launcher-1630-thread-2) [n:127.0.0.1:44610_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.DocValuesNotIndexedTest_8080C71C649636E9-001/tempDir-001/node1/.
   [junit4]   2> 1159200 INFO  (SUITE-DocValuesNotIndexedTest-seed#[8080C71C649636E9]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1159202 INFO  (SUITE-DocValuesNotIndexedTest-seed#[8080C71C649636E9]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:58011/solr ready
   [junit4]   2> 1159214 INFO  (qtp1016239370-10147) [n:127.0.0.1:55150_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&maxShardsPerNode=2&name=dv_coll&action=CREATE&numShards=4&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1159218 INFO  (OverseerThreadFactory-4301-thread-1-processing-n:127.0.0.1:44610_solr) [n:127.0.0.1:44610_solr    ] o.a.s.c.CreateCollectionCmd Create collection dv_coll
   [junit4]   2> 1159326 INFO  (qtp769838975-10118) [n:127.0.0.1:44610_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=dv_coll_shard3_replica1&action=CREATE&numShards=4&collection=dv_coll&shard=shard3&wt=javabin&version=2
   [junit4]   2> 1159327 INFO  (qtp1016239370-10148) [n:127.0.0.1:55150_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=dv_coll_shard4_replica1&action=CREATE&numShards=4&collection=dv_coll&shard=shard4&wt=javabin&version=2
   [junit4]   2> 1159329 INFO  (qtp769838975-10125) [n:127.0.0.1:44610_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=dv_coll_shard1_replica1&action=CREATE&numShards=4&collection=dv_coll&shard=shard1&wt=javabin&version=2
   [junit4]   2> 1159328 INFO  (qtp1016239370-10150) [n:127.0.0.1:55150_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=dv_coll_shard2_replica1&action=CREATE&numShards=4&collection=dv_coll&shard=shard2&wt=javabin&version=2
   [junit4]   2> 1159448 INFO  (zkCallback-1642-thread-1-processing-n:127.0.0.1:55150_solr) [n:127.0.0.1:55150_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/dv_coll/state.json] for collection [dv_coll] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1159448 INFO  (zkCallback-1636-thread-3-processing-n:127.0.0.1:44610_solr) [n:127.0.0.1:44610_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/dv_coll/state.json] for collection [dv_coll] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1160345 INFO  (qtp1016239370-10150) [n:127.0.0.1:55150_solr c:dv_coll s:shard2  x:dv_coll_shard2_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 1160345 INFO  (qtp769838975-10125) [n:127.0.0.1:44610_solr c:dv_coll s:shard1  x:dv_coll_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 1160348 INFO  (qtp1016239370-10148) [n:127.0.0.1:55150_solr c:dv_coll s:shard4  x:dv_coll_shard4_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 1160355 INFO  (qtp769838975-10118) [n:127.0.0.1:44610_solr c:dv_coll s:shard3  x:dv_coll_shard3_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 1160356 INFO  (qtp1016239370-10150) [n:127.0.0.1:55150_solr c:dv_coll s:shard2  x:dv_coll_shard2_replica1] o.a.s.s.IndexSchema [dv_coll_shard2_replica1] Schema name=minimal
   [junit4]   2> 1160357 INFO  (qtp769838975-10125) [n:127.0.0.1:44610_solr c:dv_coll s:shard1  x:dv_coll_shard1_replica1] o.a.s.s.IndexSchema [dv_coll_shard1_replica1] Schema name=minimal
   [junit4]   2> 1160359 INFO  (qtp1016239370-10150) [n:127.0.0.1:55150_solr c:dv_coll s:shard2  x:dv_coll_shard2_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1160359 INFO  (qtp769838975-10125) [n:127.0.0.1:44610_solr c:dv_coll s:shard1  x:dv_coll_shard1_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1160360 INFO  (qtp1016239370-10150) [n:127.0.0.1:55150_solr c:dv_coll s:shard2  x:dv_coll_shard2_replica1] o.a.s.c.CoreContainer Creating SolrCore 'dv_coll_shard2_replica1' using configuration from collection dv_coll
   [junit4]   2> 1160360 INFO  (qtp769838975-10125) [n:127.0.0.1:44610_solr c:dv_coll s:shard1  x:dv_coll_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'dv_coll_shard1_replica1' using configuration from collection dv_coll
   [junit4]   2> 1160361 INFO  (qtp769838975-10125) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.c.SolrCore [[dv_coll_shard1_replica1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.DocValuesNotIndexedTest_8080C71C649636E9-001/tempDir-001/node1/dv_coll_shard1_replica1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.DocValuesNotIndexedTest_8080C71C649636E9-001/tempDir-001/node1/./dv_coll_shard1_replica1/data/]
   [junit4]   2> 1160361 INFO  (qtp1016239370-10150) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.c.SolrCore [[dv_coll_shard2_replica1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.DocValuesNotIndexedTest_8080C71C649636E9-001/tempDir-001/node2/dv_coll_shard2_replica1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.DocValuesNotIndexedTest_8080C71C649636E9-001/tempDir-001/node2/./dv_coll_shard2_replica1/data/]
   [junit4]   2> 1160362 INFO  (qtp1016239370-10148) [n:127.0.0.1:55150_solr c:dv_coll s:shard4  x:dv_coll_shard4_replica1] o.a.s.s.IndexSchema [dv_coll_shard4_replica1] Schema name=minimal
   [junit4]   2> 1160365 INFO  (qtp1016239370-10148) [n:127.0.0.1:55150_solr c:dv_coll s:shard4  x:dv_coll_shard4_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1160381 INFO  (qtp769838975-10118) [n:127.0.0.1:44610_solr c:dv_coll s:shard3  x:dv_coll_shard3_replica1] o.a.s.s.IndexSchema [dv_coll_shard3_replica1] Schema name=minimal
   [junit4]   2> 1160383 INFO  (qtp1016239370-10148) [n:127.0.0.1:55150_solr c:dv_coll s:shard4  x:dv_coll_shard4_replica1] o.a.s.c.CoreContainer Creating SolrCore 'dv_coll_shard4_replica1' using configuration from collection dv_coll
   [junit4]   2> 1160383 INFO  (qtp769838975-10118) [n:127.0.0.1:44610_solr c:dv_coll s:shard3  x:dv_coll_shard3_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1160384 INFO  (qtp1016239370-10148) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.c.SolrCore [[dv_coll_shard4_replica1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.DocValuesNotIndexedTest_8080C71C649636E9-001/tempDir-001/node2/dv_coll_shard4_replica1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.DocValuesNotIndexedTest_8080C71C649636E9-001/tempDir-001/node2/./dv_coll_shard4_replica1/data/]
   [junit4]   2> 1160384 INFO  (qtp769838975-10118) [n:127.0.0.1:44610_solr c:dv_coll s:shard3  x:dv_coll_shard3_replica1] o.a.s.c.CoreContainer Creating SolrCore 'dv_coll_shard3_replica1' using configuration from collection dv_coll
   [junit4]   2> 1160385 INFO  (qtp769838975-10118) [n:127.0.0.1:44610_solr c:dv_coll s:shard3 r:core_node3 x:dv_coll_shard3_replica1] o.a.s.c.SolrCore [[dv_coll_shard3_replica1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.DocValuesNotIndexedTest_8080C71C649636E9-001/tempDir-001/node1/dv_coll_shard3_replica1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.DocValuesNotIndexedTest_8080C71C649636E9-001/tempDir-001/node1/./dv_coll_shard3_replica1/data/]
   [junit4]   2> 1160386 INFO  (qtp769838975-10125) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1160386 INFO  (qtp769838975-10125) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1160388 INFO  (qtp1016239370-10150) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1160388 INFO  (qtp1016239370-10150) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1160389 INFO  (qtp769838975-10125) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1160389 INFO  (qtp769838975-10125) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1160390 INFO  (qtp769838975-10125) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@193baaa6[dv_coll_shard1_replica1] main]
   [junit4]   2> 1160391 INFO  (qtp1016239370-10150) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1160391 INFO  (qtp1016239370-10150) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1160392 INFO  (qtp1016239370-10150) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@465fe47[dv_coll_shard2_replica1] main]
   [junit4]   2> 1160394 INFO  (qtp769838975-10125) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1160394 INFO  (qtp769838975-10125) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1160394 INFO  (qtp769838975-10125) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/conf1/managed-schema
   [junit4]   2> 1160395 INFO  (qtp769838975-10125) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1160395 INFO  (qtp769838975-10125) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1552118165841903616
   [junit4]   2> 1160397 INFO  (qtp1016239370-10148) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1160397 INFO  (qtp1016239370-10148) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1160400 INFO  (qtp1016239370-10148) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1160400 INFO  (qtp1016239370-10148) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1160400 INFO  (qtp1016239370-10148) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7dc6e8a4[dv_coll_shard4_replica1] main]
   [junit4]   2> 1160402 INFO  (searcherExecutor-4311-thread-1-processing-n:127.0.0.1:44610_solr x:dv_coll_shard1_replica1 s:shard1 c:dv_coll r:core_node1) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.c.SolrCore [dv_coll_shard1_replica1] Registered new searcher Searcher@193baaa6[dv_coll_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1160402 INFO  (qtp1016239370-10148) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1160402 INFO  (qtp1016239370-10150) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1160403 INFO  (qtp1016239370-10148) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1160403 INFO  (qtp1016239370-10148) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/conf1/managed-schema
   [junit4]   2> 1160403 INFO  (qtp1016239370-10150) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1160403 INFO  (qtp1016239370-10150) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/conf1/managed-schema
   [junit4]   2> 1160403 INFO  (qtp769838975-10118) [n:127.0.0.1:44610_solr c:dv_coll s:shard3 r:core_node3 x:dv_coll_shard3_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1160403 INFO  (qtp769838975-10118) [n:127.0.0.1:44610_solr c:dv_coll s:shard3 r:core_node3 x:dv_coll_shard3_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1160404 INFO  (qtp1016239370-10148) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1160404 INFO  (qtp1016239370-10150) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1160404 INFO  (qtp1016239370-10148) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1552118165851340800
   [junit4]   2> 1160404 INFO  (qtp1016239370-10150) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1552118165851340800
   [junit4]   2> 1160406 INFO  (qtp769838975-10118) [n:127.0.0.1:44610_solr c:dv_coll s:shard3 r:core_node3 x:dv_coll_shard3_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1160406 INFO  (qtp769838975-10118) [n:127.0.0.1:44610_solr c:dv_coll s:shard3 r:core_node3 x:dv_coll_shard3_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1160406 INFO  (searcherExecutor-4312-thread-1-processing-n:127.0.0.1:55150_solr x:dv_coll_shard4_replica1 s:shard4 c:dv_coll r:core_node2) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.c.SolrCore [dv_coll_shard4_replica1] Registered new searcher Searcher@7dc6e8a4[dv_coll_shard4_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1160406 INFO  (qtp769838975-10118) [n:127.0.0.1:44610_solr c:dv_coll s:shard3 r:core_node3 x:dv_coll_shard3_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1b7e1d55[dv_coll_shard3_replica1] main]
   [junit4]   2> 1160406 INFO  (searcherExecutor-4310-thread-1-processing-n:127.0.0.1:55150_solr x:dv_coll_shard2_replica1 s:shard2 c:dv_coll r:core_node4) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.c.SolrCore [dv_coll_shard2_replica1] Registered new searcher Searcher@465fe47[dv_coll_shard2_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1160409 INFO  (qtp769838975-10125) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1160409 INFO  (qtp769838975-10125) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1160409 INFO  (qtp769838975-10125) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:44610/solr/dv_coll_shard1_replica1/
   [junit4]   2> 1160409 INFO  (qtp769838975-10125) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1160409 INFO  (qtp769838975-10125) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.c.SyncStrategy http://127.0.0.1:44610/solr/dv_coll_shard1_replica1/ has no replicas
   [junit4]   2> 1160410 INFO  (qtp769838975-10118) [n:127.0.0.1:44610_solr c:dv_coll s:shard3 r:core_node3 x:dv_coll_shard3_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1160410 INFO  (qtp769838975-10118) [n:127.0.0.1:44610_solr c:dv_coll s:shard3 r:core_node3 x:dv_coll_shard3_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1160411 INFO  (qtp769838975-10118) [n:127.0.0.1:44610_solr c:dv_coll s:shard3 r:core_node3 x:dv_coll_shard3_replica1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/conf1/managed-schema
   [junit4]   2> 1160411 INFO  (qtp769838975-10118) [n:127.0.0.1:44610_solr c:dv_coll s:shard3 r:core_node3 x:dv_coll_shard3_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1160412 INFO  (searcherExecutor-4315-thread-1-processing-n:127.0.0.1:44610_solr x:dv_coll_shard3_replica1 s:shard3 c:dv_coll r:core_node3) [n:127.0.0.1:44610_solr c:dv_coll s:shard3 r:core_node3 x:dv_coll_shard3_replica1] o.a.s.c.SolrCore [dv_coll_shard3_replica1] Registered new searcher Searcher@1b7e1d55[dv_coll_shard3_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1160412 INFO  (qtp769838975-10118) [n:127.0.0.1:44610_solr c:dv_coll s:shard3 r:core_node3 x:dv_coll_shard3_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1552118165859729408
   [junit4]   2> 1160417 INFO  (qtp769838975-10125) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:44610/solr/dv_coll_shard1_replica1/ shard1
   [junit4]   2> 1160418 INFO  (qtp1016239370-10148) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1160418 INFO  (qtp1016239370-10148) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1160418 INFO  (qtp1016239370-10148) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:55150/solr/dv_coll_shard4_replica1/
   [junit4]   2> 1160418 INFO  (qtp1016239370-10148) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1160418 INFO  (qtp1016239370-10148) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.c.SyncStrategy http://127.0.0.1:55150/solr/dv_coll_shard4_replica1/ has no replicas
   [junit4]   2> 1160419 INFO  (qtp1016239370-10150) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1160419 INFO  (qtp1016239370-10150) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1160419 INFO  (qtp1016239370-10150) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:55150/solr/dv_coll_shard2_replica1/
   [junit4]   2> 1160419 INFO  (qtp1016239370-10150) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1160420 INFO  (qtp1016239370-10150) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.c.SyncStrategy http://127.0.0.1:55150/solr/dv_coll_shard2_replica1/ has no replicas
   [junit4]   2> 1160423 INFO  (qtp769838975-10118) [n:127.0.0.1:44610_solr c:dv_coll s:shard3 r:core_node3 x:dv_coll_shard3_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1160423 INFO  (qtp769838975-10118) [n:127.0.0.1:44610_solr c:dv_coll s:shard3 r:core_node3 x:dv_coll_shard3_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1160423 INFO  (qtp769838975-10118) [n:127.0.0.1:44610_solr c:dv_coll s:shard3 r:core_node3 x:dv_coll_shard3_replica1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:44610/solr/dv_coll_shard3_replica1/
   [junit4]   2> 1160423 INFO  (qtp769838975-10118) [n:127.0.0.1:44610_solr c:dv_coll s:shard3 r:core_node3 x:dv_coll_shard3_replica1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1160423 INFO  (qtp769838975-10118) [n:127.0.0.1:44610_solr c:dv_coll s:shard3 r:core_node3 x:dv_coll_shard3_replica1] o.a.s.c.SyncStrategy http://127.0.0.1:44610/solr/dv_coll_shard3_replica1/ has no replicas
   [junit4]   2> 1160425 INFO  (qtp1016239370-10148) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:55150/solr/dv_coll_shard4_replica1/ shard4
   [junit4]   2> 1160426 INFO  (qtp1016239370-10150) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:55150/solr/dv_coll_shard2_replica1/ shard2
   [junit4]   2> 1160430 INFO  (qtp769838975-10118) [n:127.0.0.1:44610_solr c:dv_coll s:shard3 r:core_node3 x:dv_coll_shard3_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:44610/solr/dv_coll_shard3_replica1/ shard3
   [junit4]   2> 1160538 INFO  (zkCallback-1642-thread-1-processing-n:127.0.0.1:55150_solr) [n:127.0.0.1:55150_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/dv_coll/state.json] for collection [dv_coll] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1160538 INFO  (zkCallback-1636-thread-1-processing-n:127.0.0.1:44610_solr) [n:127.0.0.1:44610_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/dv_coll/state.json] for collection [dv_coll] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1160569 INFO  (qtp769838975-10125) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1160572 INFO  (qtp769838975-10125) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=dv_coll_shard1_replica1&action=CREATE&numShards=4&collection=dv_coll&shard=shard1&wt=javabin&version=2} status=0 QTime=1243
   [junit4]   2> 1160576 INFO  (qtp1016239370-10148) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1160577 INFO  (qtp1016239370-10150) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1160580 INFO  (qtp1016239370-10148) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=dv_coll_shard4_replica1&action=CREATE&numShards=4&collection=dv_coll&shard=shard4&wt=javabin&version=2} status=0 QTime=1253
   [junit4]   2> 1160580 INFO  (qtp1016239370-10150) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=dv_coll_shard2_replica1&action=CREATE&numShards=4&collection=dv_coll&shard=shard2&wt=javabin&version=2} status=0 QTime=1252
   [junit4]   2> 1160581 INFO  (qtp769838975-10118) [n:127.0.0.1:44610_solr c:dv_coll s:shard3 r:core_node3 x:dv_coll_shard3_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1160584 INFO  (qtp769838975-10118) [n:127.0.0.1:44610_solr c:dv_coll s:shard3 r:core_node3 x:dv_coll_shard3_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=dv_coll_shard3_replica1&action=CREATE&numShards=4&collection=dv_coll&shard=shard3&wt=javabin&version=2} status=0 QTime=1258
   [junit4]   2> 1160590 INFO  (qtp1016239370-10147) [n:127.0.0.1:55150_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 1160692 INFO  (zkCallback-1636-thread-3-processing-n:127.0.0.1:44610_solr) [n:127.0.0.1:44610_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/dv_coll/state.json] for collection [dv_coll] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1160692 INFO  (zkCallback-1642-thread-1-processing-n:127.0.0.1:55150_solr) [n:127.0.0.1:55150_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/dv_coll/state.json] for collection [dv_coll] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1161225 INFO  (OverseerCollectionConfigSetProcessor-97007385262161923-127.0.0.1:44610_solr-n_0000000000) [n:127.0.0.1:44610_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1161591 INFO  (qtp1016239370-10147) [n:127.0.0.1:55150_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&maxShardsPerNode=2&name=dv_coll&action=CREATE&numShards=4&wt=javabin&version=2} status=0 QTime=2378
   [junit4]   2> 1161610 INFO  (qtp1016239370-10155) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.s.SchemaManager managed schema loaded . version : 0 
   [junit4]   2> 1161614 INFO  (qtp1016239370-10155) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.s.IndexSchema [dv_coll_shard2_replica1] Schema name=minimal
   [junit4]   2> 1161617 INFO  (qtp1016239370-10155) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1161641 INFO  (qtp1016239370-10155) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.c.ZkController Persisted config data to node /configs/conf1/managed-schema 
   [junit4]   2> 1161641 INFO  (zkCallback-1636-thread-3-processing-n:127.0.0.1:44610_solr) [n:127.0.0.1:44610_solr    ] o.a.s.s.ZkIndexSchemaReader A schema change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/configs/conf1/managed-schema, has occurred - updating schema from ZooKeeper ...
   [junit4]   2> 1161641 INFO  (zkCallback-1636-thread-1-processing-n:127.0.0.1:44610_solr) [n:127.0.0.1:44610_solr    ] o.a.s.s.ZkIndexSchemaReader A schema change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/configs/conf1/managed-schema, has occurred - updating schema from ZooKeeper ...
   [junit4]   2> 1161642 INFO  (zkCallback-1642-thread-1-processing-n:127.0.0.1:55150_solr) [n:127.0.0.1:55150_solr    ] o.a.s.s.ZkIndexSchemaReader A schema change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/configs/conf1/managed-schema, has occurred - updating schema from ZooKeeper ...
   [junit4]   2> 1161642 INFO  (zkCallback-1642-thread-2-processing-n:127.0.0.1:55150_solr) [n:127.0.0.1:55150_solr    ] o.a.s.s.ZkIndexSchemaReader A schema change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/configs/conf1/managed-schema, has occurred - updating schema from ZooKeeper ...
   [junit4]   2> 1161642 INFO  (zkCallback-1636-thread-1-processing-n:127.0.0.1:44610_solr) [n:127.0.0.1:44610_solr    ] o.a.s.s.ZkIndexSchemaReader Retrieved schema version 1 from ZooKeeper
   [junit4]   2> 1161642 INFO  (zkCallback-1636-thread-3-processing-n:127.0.0.1:44610_solr) [n:127.0.0.1:44610_solr    ] o.a.s.s.ZkIndexSchemaReader Retrieved schema version 1 from ZooKeeper
   [junit4]   2> 1161645 INFO  (zkCallback-1636-thread-1-processing-n:127.0.0.1:44610_solr) [n:127.0.0.1:44610_solr    ] o.a.s.s.IndexSchema [dv_coll_shard3_replica1] Schema name=minimal
   [junit4]   2> 1161645 INFO  (zkCallback-1642-thread-1-processing-n:127.0.0.1:55150_solr) [n:127.0.0.1:55150_solr    ] o.a.s.s.ZkIndexSchemaReader Retrieved schema version 1 from ZooKeeper
   [junit4]   2> 1161649 INFO  (qtp1016239370-10155) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.s.ManagedIndexSchema Waiting up to 599 secs for 3 replicas to apply schema update version 1 for collection dv_coll
   [junit4]   2> 1161655 INFO  (zkCallback-1636-thread-1-processing-n:127.0.0.1:44610_solr) [n:127.0.0.1:44610_solr    ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1161655 INFO  (zkCallback-1636-thread-1-processing-n:127.0.0.1:44610_solr) [n:127.0.0.1:44610_solr    ] o.a.s.s.ZkIndexSchemaReader Finished refreshing schema in 12 ms
   [junit4]   2> 1161655 INFO  (zkCallback-1642-thread-1-processing-n:127.0.0.1:55150_solr) [n:127.0.0.1:55150_solr    ] o.a.s.s.IndexSchema [dv_coll_shard4_replica1] Schema name=minimal
   [junit4]   2> 1161658 INFO  (qtp769838975-10129) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.h.SchemaHandler REFRESHING SCHEMA (refreshIfBelowVersion=1, currentVersion=0) before returning version!
   [junit4]   2> 1161658 INFO  (qtp1016239370-10156) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.h.SchemaHandler REFRESHING SCHEMA (refreshIfBelowVersion=1, currentVersion=0) before returning version!
   [junit4]   2> 1161659 INFO  (zkCallback-1636-thread-3-processing-n:127.0.0.1:44610_solr) [n:127.0.0.1:44610_solr    ] o.a.s.s.IndexSchema [dv_coll_shard1_replica1] Schema name=minimal
   [junit4]   2> 1161667 INFO  (zkCallback-1636-thread-3-processing-n:127.0.0.1:44610_solr) [n:127.0.0.1:44610_solr    ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1161667 INFO  (zkCallback-1636-thread-3-processing-n:127.0.0.1:44610_solr) [n:127.0.0.1:44610_solr    ] o.a.s.s.ZkIndexSchemaReader Finished refreshing schema in 24 ms
   [junit4]   2> 1161667 INFO  (qtp769838975-10129) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.c.S.Request [dv_coll_shard1_replica1]  webapp=/solr path=/schema/zkversion params={refreshIfBelowVersion=1&wt=javabin&version=2} status=0 QTime=9
   [junit4]   2> 1161670 INFO  (zkCallback-1642-thread-1-processing-n:127.0.0.1:55150_solr) [n:127.0.0.1:55150_solr    ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1161670 INFO  (zkCallback-1642-thread-1-processing-n:127.0.0.1:55150_solr) [n:127.0.0.1:55150_solr    ] o.a.s.s.ZkIndexSchemaReader Finished refreshing schema in 24 ms
   [junit4]   2> 1161671 INFO  (qtp1016239370-10156) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.c.S.Request [dv_coll_shard4_replica1]  webapp=/solr path=/schema/zkversion params={refreshIfBelowVersion=1&wt=javabin&version=2} status=0 QTime=12
   [junit4]   2> 1161671 INFO  (Thread-2705) [n:127.0.0.1:44610_solr    ] o.a.s.c.SolrCore config update listener called for core dv_coll_shard1_replica1
   [junit4]   2> 1161672 INFO  (Thread-2706) [n:127.0.0.1:55150_solr    ] o.a.s.c.SolrCore config update listener called for core dv_coll_shard4_replica1
   [junit4]   2> 1161674 INFO  (qtp769838975-10130) [n:127.0.0.1:44610_solr c:dv_coll s:shard3 r:core_node3 x:dv_coll_shard3_replica1] o.a.s.c.S.Request [dv_coll_shard3_replica1]  webapp=/solr path=/schema/zkversion params={refreshIfBelowVersion=1&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1161675 INFO  (qtp1016239370-10155) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.s.ManagedIndexSchema Took 26.0ms for 3 replicas to apply schema update version 1 for collection dv_coll
   [junit4]   2> 1161675 INFO  (qtp1016239370-10155) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.c.S.Request [dv_coll_shard2_replica1]  webapp=/solr path=/schema params={wt=javabin&version=2} status=0 QTime=66
   [junit4]   2> 1161680 INFO  (zkCallback-1642-thread-2-processing-n:127.0.0.1:55150_solr) [n:127.0.0.1:55150_solr    ] o.a.s.s.ZkIndexSchemaReader Retrieved schema version 1 from ZooKeeper
   [junit4]   2> 1161691 INFO  (Thread-2705) [n:127.0.0.1:44610_solr    ] o.a.s.c.SolrCore config update listener called for core dv_coll_shard3_replica1
   [junit4]   2> 1161692 INFO  (Thread-2706) [n:127.0.0.1:55150_solr    ] o.a.s.c.SolrCore config update listener called for core dv_coll_shard2_replica1
   [junit4]   2> 1161704 INFO  (TEST-DocValuesNotIndexedTest.testDistribFaceting-seed#[8080C71C649636E9]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testDistribFaceting
   [junit4]   2> 1161707 INFO  (zkCallback-1642-thread-2-processing-n:127.0.0.1:55150_solr) [n:127.0.0.1:55150_solr    ] o.a.s.s.IndexSchema [dv_coll_shard2_replica1] Schema name=minimal
   [junit4]   2> 1161707 INFO  (Thread-2706) [n:127.0.0.1:55150_solr    ] o.a.s.s.SchemaManager managed schema loaded . version : 1 
   [junit4]   2> 1161713 INFO  (zkCallback-1642-thread-2-processing-n:127.0.0.1:55150_solr) [n:127.0.0.1:55150_solr    ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1161713 INFO  (zkCallback-1642-thread-2-processing-n:127.0.0.1:55150_solr) [n:127.0.0.1:55150_solr    ] o.a.s.s.ZkIndexSchemaReader Finished refreshing schema in 33 ms
   [junit4]   2> 1161720 INFO  (Thread-2706) [n:127.0.0.1:55150_solr    ] o.a.s.s.IndexSchema [dv_coll_shard2_replica1] Schema name=minimal
   [junit4]   2> 1161724 INFO  (qtp1016239370-10157) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.u.p.LogUpdateProcessorFactory [dv_coll_shard2_replica1]  webapp=/solr path=/update params={update.distrib=TOLEADER&distrib.from=http://127.0.0.1:44610/solr/dv_coll_shard3_replica1/&wt=javabin&version=2}{deleteByQuery=*:* (-1552118167233363968)} 0 2
   [junit4]   2> 1161726 INFO  (qtp1016239370-10157) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.u.p.LogUpdateProcessorFactory [dv_coll_shard4_replica1]  webapp=/solr path=/update params={update.distrib=TOLEADER&distrib.from=http://127.0.0.1:44610/solr/dv_coll_shard3_replica1/&wt=javabin&version=2}{deleteByQuery=*:* (-1552118167235461120)} 0 1
   [junit4]   2> 1161728 INFO  (qtp769838975-10123) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [dv_coll_shard1_replica1]  webapp=/solr path=/update params={update.distrib=TOLEADER&distrib.from=http://127.0.0.1:44610/solr/dv_coll_shard3_replica1/&wt=javabin&version=2}{deleteByQuery=*:* (-1552118167237558272)} 0 2
   [junit4]   2> 1161729 INFO  (qtp769838975-10130) [n:127.0.0.1:44610_solr c:dv_coll s:shard3 r:core_node3 x:dv_coll_shard3_replica1] o.a.s.u.p.LogUpdateProcessorFactory [dv_coll_shard3_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{deleteByQuery=*:* (-1552118167224975360)} 0 15
   [junit4]   2> 1161731 INFO  (qtp769838975-10121) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1161731 INFO  (qtp769838975-10121) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 1161733 INFO  (qtp769838975-10124) [n:127.0.0.1:44610_solr c:dv_coll s:shard3 r:core_node3 x:dv_coll_shard3_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1161734 INFO  (qtp769838975-10124) [n:127.0.0.1:44610_solr c:dv_coll s:shard3 r:core_node3 x:dv_coll_shard3_replica1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 1161738 INFO  (Thread-2706) [n:127.0.0.1:55150_solr    ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1161738 INFO  (Thread-2706) [n:127.0.0.1:55150_solr    ] o.a.s.c.SolrCore core reload dv_coll_shard2_replica1
   [junit4]   2> 1161738 INFO  (qtp769838975-10121) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2e3b40de[dv_coll_shard1_replica1] main]
   [junit4]   2> 1161739 INFO  (qtp769838975-10124) [n:127.0.0.1:44610_solr c:dv_coll s:shard3 r:core_node3 x:dv_coll_shard3_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3c6fc3ac[dv_coll_shard3_replica1] main]
   [junit4]   2> 1161739 INFO  (qtp769838975-10124) [n:127.0.0.1:44610_solr c:dv_coll s:shard3 r:core_node3 x:dv_coll_shard3_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1161739 INFO  (qtp769838975-10121) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1161739 INFO  (searcherExecutor-4315-thread-1-processing-n:127.0.0.1:44610_solr x:dv_coll_shard3_replica1 s:shard3 c:dv_coll r:core_node3) [n:127.0.0.1:44610_solr c:dv_coll s:shard3 r:core_node3 x:dv_coll_shard3_replica1] o.a.s.c.SolrCore [dv_coll_shard3_replica1] Registered new searcher Searcher@3c6fc3ac[dv_coll_shard3_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1161739 INFO  (qtp1016239370-10144) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1161739 INFO  (qtp1016239370-10144) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 1161740 INFO  (searcherExecutor-4311-thread-1-processing-n:127.0.0.1:44610_solr x:dv_coll_shard1_replica1 s:shard1 c:dv_coll r:core_node1) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.c.SolrCore [dv_coll_shard1_replica1] Registered new searcher Searcher@2e3b40de[dv_coll_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1161740 INFO  (qtp1016239370-10144) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@61a2601d[dv_coll_shard4_replica1] main]
   [junit4]   2> 1161740 INFO  (qtp769838975-10121) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [dv_coll_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:55150/solr/dv_coll_shard4_replica1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 8
   [junit4]   2> 1161740 INFO  (qtp1016239370-10144) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1161740 INFO  (searcherExecutor-4312-thread-1-processing-n:127.0.0.1:55150_solr x:dv_coll_shard4_replica1 s:shard4 c:dv_coll r:core_node2) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.c.SolrCore [dv_coll_shard4_replica1] Registered new searcher Searcher@61a2601d[dv_coll_shard4_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1161740 INFO  (qtp1016239370-10144) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.u.p.LogUpdateProcessorFactory [dv_coll_shard4_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:55150/solr/dv_coll_shard4_replica1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 1
   [junit4]   2> 1161740 INFO  (qtp769838975-10124) [n:127.0.0.1:44610_solr c:dv_coll s:shard3 r:core_node3 x:dv_coll_shard3_replica1] o.a.s.u.p.LogUpdateProcessorFactory [dv_coll_shard3_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:55150/solr/dv_coll_shard4_replica1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 7
   [junit4]   2> 1161741 INFO  (qtp1016239370-10148) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1161741 INFO  (qtp1016239370-10148) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 1161742 INFO  (qtp1016239370-10148) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@445a9df5[dv_coll_shard2_replica1] main]
   [junit4]   2> 1161742 INFO  (qtp1016239370-10148) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1161742 INFO  (searcherExecutor-4310-thread-1-processing-n:127.0.0.1:55150_solr x:dv_coll_shard2_replica1 s:shard2 c:dv_coll r:core_node4) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.c.SolrCore [dv_coll_shard2_replica1] Registered new searcher Searcher@445a9df5[dv_coll_shard2_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1161742 INFO  (qtp1016239370-10148) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.u.p.LogUpdateProcessorFactory [dv_coll_shard2_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:55150/solr/dv_coll_shard4_replica1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 2
   [junit4]   2> 1161742 INFO  (qtp1016239370-10151) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.u.p.LogUpdateProcessorFactory [dv_coll_shard4_replica1]  webapp=/solr path=/update params={_stateVer_=dv_coll:3&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 12
   [junit4]   2> 1161750 INFO  (qtp769838975-10125) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [dv_coll_shard1_replica1]  webapp=/solr path=/update params={_stateVer_=dv_coll:3&wt=javabin&version=2}{add=[1 (1552118167260626944)]} 0 2
   [junit4]   2> 1161752 INFO  (qtp1016239370-10144) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1161752 INFO  (qtp1016239370-10144) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 1161752 INFO  (qtp1016239370-10144) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1161753 INFO  (qtp1016239370-10144) [n:127.0.0.1:55150_solr c:dv_coll s:shard2 r:core_node4 x:dv_coll_shard2_replica1] o.a.s.u.p.LogUpdateProcessorFactory [dv_coll_shard2_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:44610/solr/dv_coll_shard1_replica1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 0
   [junit4]   2> 1161754 INFO  (qtp769838975-10129) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1161754 INFO  (qtp769838975-10129) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@6c62044b
   [junit4]   2> 1161759 INFO  (qtp1016239370-10147) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1161759 INFO  (qtp1016239370-10147) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 1161760 INFO  (qtp769838975-10131) [n:127.0.0.1:44610_solr c:dv_coll s:shard3 r:core_node3 x:dv_coll_shard3_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1161760 INFO  (qtp769838975-10131) [n:127.0.0.1:44610_solr c:dv_coll s:shard3 r:core_node3 x:dv_coll_shard3_replica1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 1161760 INFO  (qtp769838975-10131) [n:127.0.0.1:44610_solr c:dv_coll s:shard3 r:core_node3 x:dv_coll_shard3_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1161761 INFO  (qtp769838975-10131) [n:127.0.0.1:44610_solr c:dv_coll s:shard3 r:core_node3 x:dv_coll_shard3_replica1] o.a.s.u.p.LogUpdateProcessorFactory [dv_coll_shard3_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:44610/solr/dv_coll_shard1_replica1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 1
   [junit4]   2> 1161767 INFO  (Thread-2706) [n:127.0.0.1:55150_solr    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 1161783 INFO  (Thread-2706) [n:127.0.0.1:55150_solr    ] o.a.s.s.IndexSchema [dv_coll_shard2_replica1] Schema name=minimal
   [junit4]   2> 1161784 INFO  (qtp1016239370-10147) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1161784 INFO  (qtp1016239370-10147) [n:127.0.0.1:55150_solr c:dv_coll s:shard4 r:core_node2 x:dv_coll_shard4_replica1] o.a.s.u.p.LogUpdateProcessorFactory [dv_coll_shard4_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:44610/solr/dv_coll_shard1_replica1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 25
   [junit4]   2> 1161787 INFO  (qtp769838975-10129) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@22d94766[dv_coll_shard1_replica1] main]
   [junit4]   2> 1161787 INFO  (qtp769838975-10129) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1161788 INFO  (searcherExecutor-4311-thread-1-processing-n:127.0.0.1:44610_solr x:dv_coll_shard1_replica1 s:shard1 c:dv_coll r:core_node1) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.c.SolrCore [dv_coll_shard1_replica1] Registered new searcher Searcher@22d94766[dv_coll_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C1)))}
   [junit4]   2> 1161792 INFO  (qtp769838975-10129) [n:127.0.0.1:44610_solr c:dv_coll s:shard1 r:core_node1 x:dv_coll_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [dv_coll_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:44610/solr/dv_coll_shard1_replica1/&commi

[...truncated too long message...]

ldCacheSort, SchemaVersionSpecificBehaviorTest, RulesTest, AnalysisAfterCoreReloadTest, TestConfigOverlay, CustomCollectionTest, ZkControllerTest, TestRemoteStreaming, TestManagedSynonymFilterFactory, ConcurrentDeleteAndCreateCollectionTest, XsltUpdateRequestHandlerTest, CdcrUpdateLogTest, SolrRequestParserTest, TestWordDelimiterFilterFactory, TestReloadDeadlock, RestartWhileUpdatingTest, TestAnalyzeInfixSuggestions, TestCryptoKeys, TestCSVResponseWriter, DocumentBuilderTest, XmlUpdateRequestHandlerTest, ClassificationUpdateProcessorIntegrationTest, BasicDistributedZkTest, ChaosMonkeySafeLeaderTest, CollectionsAPIDistributedZkTest, SyncSliceTest, LeaderElectionIntegrationTest, BasicZkTest, RecoveryZkTest, FullSolrCloudDistribCmdsTest, TestReplicationHandler, LeaderElectionTest, TestZkChroot, TestRecovery, TestRealTimeGet, TestJoin, DistributedTermsComponentTest, StatsComponentTest, TestFiltering, SoftAutoCommitTest, SuggesterTSTTest, TestStressLucene, SpatialFilterTest, SuggesterWFSTTest, NoCacheHeaderTest, TestPseudoReturnFields, TestUpdate, FieldMutatingUpdateProcessorTest, QueryEqualityTest, DistanceFunctionTest, DebugComponentTest, CacheHeaderTest, LukeRequestHandlerTest, TestReversedWildcardFilterFactory, TestQueryTypes, TestSolrQueryParser, SolrPluginUtilsTest, UpdateRequestProcessorFactoryTest, JsonLoaderTest, PingRequestHandlerTest, TestPhraseSuggestions, TestCodecSupport, TestPerFieldSimilarity, TestIBSimilarityFactory, TestJmxMonitoredMap, PluginInfoTest, ChaosMonkeyNothingIsSafeTest, CdcrReplicationDistributedZkTest, CdcrVersionReplicationTest, CleanupOldIndexTest, CloudExitableDirectoryReaderTest, CollectionStateFormat2Test, CollectionTooManyReplicasTest, CollectionsAPIAsyncDistributedZkTest, CollectionsAPISolrJTest, DeleteReplicaTest, DistribCursorPagingTest, DistribDocExpirationUpdateProcessorTest, HttpPartitionTest]
   [junit4] Completed [466/658 (3!)] on J0 in 125.22s, 1 test, 1 error <<< FAILURES!

[...truncated 11771 lines...]
   [junit4] Suite: org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/contrib/solr-uima/test/J0/temp/solr.uima.processor.UIMAUpdateRequestProcessorTest_3D0313FFFB63451D-001/init-core-data-001
   [junit4]   2> log4j:WARN No appenders could be found for logger (org.eclipse.jetty.util.log).
   [junit4]   2> log4j:WARN Please initialize the log4j system properly.
   [junit4]   2> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
   [junit4]   2> ????. 27, 2016 8:37:50 AM WhitespaceTokenizer initialize
   [junit4]   2> INFO: "Whitespace tokenizer successfully initialized"
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=UIMAUpdateRequestProcessorTest -Dtests.method=testErrorHandling -Dtests.seed=3D0313FFFB63451D -Dtests.slow=true -Dtests.locale=mk -Dtests.timezone=Indian/Maldives -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   6.52s J0 | UIMAUpdateRequestProcessorTest.testErrorHandling <<<
   [junit4]    > Throwable #1: org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
   [junit4]    >   Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([3D0313FFFB63451D:8F76DC1ADB5C4C9]:0)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
   [junit4]    > 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testErrorHandling(UIMAUpdateRequestProcessorTest.java:137)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=UIMAUpdateRequestProcessorTest -Dtests.method=testMultiplierProcessing -Dtests.seed=3D0313FFFB63451D -Dtests.slow=true -Dtests.locale=mk -Dtests.timezone=Indian/Maldives -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.04s J0 | UIMAUpdateRequestProcessorTest.testMultiplierProcessing <<<
   [junit4]    > Throwable #1: org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
   [junit4]    >   Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([3D0313FFFB63451D:602D952FC3769E9]:0)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
   [junit4]    > 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing(UIMAUpdateRequestProcessorTest.java:83)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testMultiplierProcessing(UIMAUpdateRequestProcessorTest.java:177)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> ????. 27, 2016 8:37:54 AM WhitespaceTokenizer initialize
   [junit4]   2> INFO: "Whitespace tokenizer successfully initialized"
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=UIMAUpdateRequestProcessorTest -Dtests.method=testProcessing -Dtests.seed=3D0313FFFB63451D -Dtests.slow=true -Dtests.locale=mk -Dtests.timezone=Indian/Maldives -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.01s J0 | UIMAUpdateRequestProcessorTest.testProcessing <<<
   [junit4]    > Throwable #1: org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
   [junit4]    >   Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([3D0313FFFB63451D:2E8D178DBC90C4B7]:0)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
   [junit4]    > 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing(UIMAUpdateRequestProcessorTest.java:83)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=UIMAUpdateRequestProcessorTest -Dtests.method=testTwoUpdates -Dtests.seed=3D0313FFFB63451D -Dtests.slow=true -Dtests.locale=mk -Dtests.timezone=Indian/Maldives -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.01s J0 | UIMAUpdateRequestProcessorTest.testTwoUpdates <<<
   [junit4]    > Throwable #1: org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">1</field><field name="text">The Apache Software Foundation is happy to announce BarCampApache Sydney, Australia, the first ASF-backed event in the Southern Hemisphere!</field></doc></add>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([3D0313FFFB63451D:304944A0E34C27B3]:0)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
   [junit4]    > 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testTwoUpdates(UIMAUpdateRequestProcessorTest.java:101)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/contrib/solr-uima/test/J0/temp/solr.uima.processor.UIMAUpdateRequestProcessorTest_3D0313FFFB63451D-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1373, maxMBSortInHeap=7.919182768184008, sim=RandomSimilarity(queryNorm=true): {}, locale=mk, timezone=Indian/Maldives
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=3,threads=1,free=234199904,total=508559360
   [junit4]   2> NOTE: All tests run in this JVM: [UIMAUpdateRequestProcessorTest]
   [junit4] Completed [2/2 (1!)] on J0 in 13.91s, 6 tests, 4 errors <<< FAILURES!

[...truncated 51250 lines...]


[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 972 - Still Unstable!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/972/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC

4 tests failed.
FAILED:  org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing

Error Message:
unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes.    Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>

Stack Trace:
org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
  Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
	at __randomizedtesting.SeedInfo.seed([61D00DF9E777594A:725E098BA084D8E0]:0)
	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing(UIMAUpdateRequestProcessorTest.java:83)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testMultiplierProcessing

Error Message:
unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes.    Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>

Stack Trace:
org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
  Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
	at __randomizedtesting.SeedInfo.seed([61D00DF9E777594A:5AD1C754E02375BE]:0)
	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing(UIMAUpdateRequestProcessorTest.java:83)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testMultiplierProcessing(UIMAUpdateRequestProcessorTest.java:177)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testErrorHandling

Error Message:
unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes.    Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>

Stack Trace:
org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
  Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
	at __randomizedtesting.SeedInfo.seed([61D00DF9E777594A:542473C7B1A1D89E]:0)
	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testErrorHandling(UIMAUpdateRequestProcessorTest.java:137)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testTwoUpdates

Error Message:
unknown UpdateRequestProcessorChain: <add><doc><field name="id">1</field><field name="text">The Apache Software Foundation is happy to announce BarCampApache Sydney, Australia, the first ASF-backed event in the Southern Hemisphere!</field></doc></add>

Stack Trace:
org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">1</field><field name="text">The Apache Software Foundation is happy to announce BarCampApache Sydney, Australia, the first ASF-backed event in the Southern Hemisphere!</field></doc></add>
	at __randomizedtesting.SeedInfo.seed([61D00DF9E777594A:6C9A5AA6FF583BE4]:0)
	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testTwoUpdates(UIMAUpdateRequestProcessorTest.java:101)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 23816 lines...]
   [junit4] Suite: org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/contrib/solr-uima/test/J0/temp/solr.uima.processor.UIMAUpdateRequestProcessorTest_61D00DF9E777594A-001/init-core-data-001
   [junit4]   2> log4j:WARN No appenders could be found for logger (org.eclipse.jetty.util.log).
   [junit4]   2> log4j:WARN Please initialize the log4j system properly.
   [junit4]   2> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
   [junit4]   2> Kas 26, 2016 6:27:30 PM WhitespaceTokenizer initialize
   [junit4]   2> INFO: "Whitespace tokenizer successfully initialized"
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=UIMAUpdateRequestProcessorTest -Dtests.method=testProcessing -Dtests.seed=61D00DF9E777594A -Dtests.slow=true -Dtests.locale=tr -Dtests.timezone=Etc/GMT-2 -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   5.86s J0 | UIMAUpdateRequestProcessorTest.testProcessing <<<
   [junit4]    > Throwable #1: org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
   [junit4]    >   Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([61D00DF9E777594A:725E098BA084D8E0]:0)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
   [junit4]    > 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing(UIMAUpdateRequestProcessorTest.java:83)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=UIMAUpdateRequestProcessorTest -Dtests.method=testMultiplierProcessing -Dtests.seed=61D00DF9E777594A -Dtests.slow=true -Dtests.locale=tr -Dtests.timezone=Etc/GMT-2 -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.02s J0 | UIMAUpdateRequestProcessorTest.testMultiplierProcessing <<<
   [junit4]    > Throwable #1: org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
   [junit4]    >   Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([61D00DF9E777594A:5AD1C754E02375BE]:0)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
   [junit4]    > 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing(UIMAUpdateRequestProcessorTest.java:83)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testMultiplierProcessing(UIMAUpdateRequestProcessorTest.java:177)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Kas 26, 2016 6:27:34 PM WhitespaceTokenizer initialize
   [junit4]   2> INFO: "Whitespace tokenizer successfully initialized"
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=UIMAUpdateRequestProcessorTest -Dtests.method=testErrorHandling -Dtests.seed=61D00DF9E777594A -Dtests.slow=true -Dtests.locale=tr -Dtests.timezone=Etc/GMT-2 -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.02s J0 | UIMAUpdateRequestProcessorTest.testErrorHandling <<<
   [junit4]    > Throwable #1: org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
   [junit4]    >   Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([61D00DF9E777594A:542473C7B1A1D89E]:0)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
   [junit4]    > 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testErrorHandling(UIMAUpdateRequestProcessorTest.java:137)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=UIMAUpdateRequestProcessorTest -Dtests.method=testTwoUpdates -Dtests.seed=61D00DF9E777594A -Dtests.slow=true -Dtests.locale=tr -Dtests.timezone=Etc/GMT-2 -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.01s J0 | UIMAUpdateRequestProcessorTest.testTwoUpdates <<<
   [junit4]    > Throwable #1: org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">1</field><field name="text">The Apache Software Foundation is happy to announce BarCampApache Sydney, Australia, the first ASF-backed event in the Southern Hemisphere!</field></doc></add>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([61D00DF9E777594A:6C9A5AA6FF583BE4]:0)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
   [junit4]    > 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testTwoUpdates(UIMAUpdateRequestProcessorTest.java:101)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/contrib/solr-uima/test/J0/temp/solr.uima.processor.UIMAUpdateRequestProcessorTest_61D00DF9E777594A-001
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=true): {}, locale=tr, timezone=Etc/GMT-2
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=3,threads=1,free=137976928,total=486580224
   [junit4]   2> NOTE: All tests run in this JVM: [UIMAUpdateRequestProcessorTest]
   [junit4] Completed [2/2 (1!)] on J0 in 13.97s, 6 tests, 4 errors <<< FAILURES!

[...truncated 51251 lines...]


[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 971 - Still Unstable!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/971/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC

4 tests failed.
FAILED:  org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testTwoUpdates

Error Message:
unknown UpdateRequestProcessorChain: <add><doc><field name="id">1</field><field name="text">The Apache Software Foundation is happy to announce BarCampApache Sydney, Australia, the first ASF-backed event in the Southern Hemisphere!</field></doc></add>

Stack Trace:
org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">1</field><field name="text">The Apache Software Foundation is happy to announce BarCampApache Sydney, Australia, the first ASF-backed event in the Southern Hemisphere!</field></doc></add>
	at __randomizedtesting.SeedInfo.seed([9C245C59CBCCCB77:916E0B06D3E3A9D9]:0)
	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testTwoUpdates(UIMAUpdateRequestProcessorTest.java:101)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing

Error Message:
unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes.    Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>

Stack Trace:
org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
  Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
	at __randomizedtesting.SeedInfo.seed([9C245C59CBCCCB77:8FAA582B8C3F4ADD]:0)
	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing(UIMAUpdateRequestProcessorTest.java:83)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testMultiplierProcessing

Error Message:
unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes.    Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>

Stack Trace:
org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
  Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
	at __randomizedtesting.SeedInfo.seed([9C245C59CBCCCB77:A72596F4CC98E783]:0)
	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing(UIMAUpdateRequestProcessorTest.java:83)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testMultiplierProcessing(UIMAUpdateRequestProcessorTest.java:177)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testErrorHandling

Error Message:
unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes.    Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>

Stack Trace:
org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
  Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
	at __randomizedtesting.SeedInfo.seed([9C245C59CBCCCB77:A9D022679D1A4AA3]:0)
	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testErrorHandling(UIMAUpdateRequestProcessorTest.java:137)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 23820 lines...]
   [junit4] Suite: org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/contrib/solr-uima/test/J0/temp/solr.uima.processor.UIMAUpdateRequestProcessorTest_9C245C59CBCCCB77-001/init-core-data-001
   [junit4]   2> log4j:WARN No appenders could be found for logger (org.eclipse.jetty.util.log).
   [junit4]   2> log4j:WARN Please initialize the log4j system properly.
   [junit4]   2> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
   [junit4]   2> Nov 26, 2016 3:54:09 PM WhitespaceTokenizer initialize
   [junit4]   2> INFO: "Whitespace tokenizer successfully initialized"
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=UIMAUpdateRequestProcessorTest -Dtests.method=testTwoUpdates -Dtests.seed=9C245C59CBCCCB77 -Dtests.slow=true -Dtests.locale=ms -Dtests.timezone=Australia/Adelaide -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   5.46s J0 | UIMAUpdateRequestProcessorTest.testTwoUpdates <<<
   [junit4]    > Throwable #1: org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">1</field><field name="text">The Apache Software Foundation is happy to announce BarCampApache Sydney, Australia, the first ASF-backed event in the Southern Hemisphere!</field></doc></add>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([9C245C59CBCCCB77:916E0B06D3E3A9D9]:0)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
   [junit4]    > 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testTwoUpdates(UIMAUpdateRequestProcessorTest.java:101)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=UIMAUpdateRequestProcessorTest -Dtests.method=testProcessing -Dtests.seed=9C245C59CBCCCB77 -Dtests.slow=true -Dtests.locale=ms -Dtests.timezone=Australia/Adelaide -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.01s J0 | UIMAUpdateRequestProcessorTest.testProcessing <<<
   [junit4]    > Throwable #1: org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
   [junit4]    >   Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([9C245C59CBCCCB77:8FAA582B8C3F4ADD]:0)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
   [junit4]    > 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing(UIMAUpdateRequestProcessorTest.java:83)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Nov 26, 2016 3:54:13 PM WhitespaceTokenizer initialize
   [junit4]   2> INFO: "Whitespace tokenizer successfully initialized"
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=UIMAUpdateRequestProcessorTest -Dtests.method=testMultiplierProcessing -Dtests.seed=9C245C59CBCCCB77 -Dtests.slow=true -Dtests.locale=ms -Dtests.timezone=Australia/Adelaide -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.01s J0 | UIMAUpdateRequestProcessorTest.testMultiplierProcessing <<<
   [junit4]    > Throwable #1: org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
   [junit4]    >   Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([9C245C59CBCCCB77:A72596F4CC98E783]:0)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
   [junit4]    > 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing(UIMAUpdateRequestProcessorTest.java:83)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testMultiplierProcessing(UIMAUpdateRequestProcessorTest.java:177)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=UIMAUpdateRequestProcessorTest -Dtests.method=testErrorHandling -Dtests.seed=9C245C59CBCCCB77 -Dtests.slow=true -Dtests.locale=ms -Dtests.timezone=Australia/Adelaide -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.02s J0 | UIMAUpdateRequestProcessorTest.testErrorHandling <<<
   [junit4]    > Throwable #1: org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
   [junit4]    >   Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([9C245C59CBCCCB77:A9D022679D1A4AA3]:0)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
   [junit4]    > 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testErrorHandling(UIMAUpdateRequestProcessorTest.java:137)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/contrib/solr-uima/test/J0/temp/solr.uima.processor.UIMAUpdateRequestProcessorTest_9C245C59CBCCCB77-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1308, maxMBSortInHeap=5.057588981572463, sim=RandomSimilarity(queryNorm=false): {}, locale=ms, timezone=Australia/Adelaide
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=3,threads=1,free=137420864,total=486039552
   [junit4]   2> NOTE: All tests run in this JVM: [UIMAUpdateRequestProcessorTest]
   [junit4] Completed [2/2 (1!)] on J0 in 12.65s, 6 tests, 4 errors <<< FAILURES!

[...truncated 51249 lines...]


[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 970 - Still Unstable!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/970/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC

5 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsRecoveryZkTest

Error Message:
ObjectTracker found 1 object(s) that were not released!!! [HdfsTransactionLog] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)  at org.apache.solr.update.HdfsTransactionLog.<init>(HdfsTransactionLog.java:130)  at org.apache.solr.update.HdfsUpdateLog.init(HdfsUpdateLog.java:202)  at org.apache.solr.update.UpdateHandler.<init>(UpdateHandler.java:137)  at org.apache.solr.update.UpdateHandler.<init>(UpdateHandler.java:94)  at org.apache.solr.update.DirectUpdateHandler2.<init>(DirectUpdateHandler2.java:102)  at sun.reflect.GeneratedConstructorAccessor178.newInstance(Unknown Source)  at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)  at java.lang.reflect.Constructor.newInstance(Constructor.java:423)  at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:723)  at org.apache.solr.core.SolrCore.createUpdateHandler(SolrCore.java:785)  at org.apache.solr.core.SolrCore.initUpdateHandler(SolrCore.java:1024)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:889)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:793)  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:868)  at org.apache.solr.core.CoreContainer.lambda$load$0(CoreContainer.java:517)  at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)  at java.lang.Thread.run(Thread.java:745)  

Stack Trace:
java.lang.AssertionError: ObjectTracker found 1 object(s) that were not released!!! [HdfsTransactionLog]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
	at org.apache.solr.update.HdfsTransactionLog.<init>(HdfsTransactionLog.java:130)
	at org.apache.solr.update.HdfsUpdateLog.init(HdfsUpdateLog.java:202)
	at org.apache.solr.update.UpdateHandler.<init>(UpdateHandler.java:137)
	at org.apache.solr.update.UpdateHandler.<init>(UpdateHandler.java:94)
	at org.apache.solr.update.DirectUpdateHandler2.<init>(DirectUpdateHandler2.java:102)
	at sun.reflect.GeneratedConstructorAccessor178.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:723)
	at org.apache.solr.core.SolrCore.createUpdateHandler(SolrCore.java:785)
	at org.apache.solr.core.SolrCore.initUpdateHandler(SolrCore.java:1024)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:889)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:793)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:868)
	at org.apache.solr.core.CoreContainer.lambda$load$0(CoreContainer.java:517)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)


	at __randomizedtesting.SeedInfo.seed([C3FB20DCFC9A64A6]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:266)
	at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testTwoUpdates

Error Message:
unknown UpdateRequestProcessorChain: <add><doc><field name="id">1</field><field name="text">The Apache Software Foundation is happy to announce BarCampApache Sydney, Australia, the first ASF-backed event in the Southern Hemisphere!</field></doc></add>

Stack Trace:
org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">1</field><field name="text">The Apache Software Foundation is happy to announce BarCampApache Sydney, Australia, the first ASF-backed event in the Southern Hemisphere!</field></doc></add>
	at __randomizedtesting.SeedInfo.seed([A4236383C33B069F:A96934DCDB146431]:0)
	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testTwoUpdates(UIMAUpdateRequestProcessorTest.java:101)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testErrorHandling

Error Message:
unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes.    Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>

Stack Trace:
org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
  Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
	at __randomizedtesting.SeedInfo.seed([A4236383C33B069F:91D71DBD95ED874B]:0)
	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testErrorHandling(UIMAUpdateRequestProcessorTest.java:137)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing

Error Message:
unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes.    Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>

Stack Trace:
org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
  Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
	at __randomizedtesting.SeedInfo.seed([A4236383C33B069F:B7AD67F184C88735]:0)
	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing(UIMAUpdateRequestProcessorTest.java:83)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testMultiplierProcessing

Error Message:
unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes.    Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>

Stack Trace:
org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
  Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
	at __randomizedtesting.SeedInfo.seed([A4236383C33B069F:9F22A92EC46F2A6B]:0)
	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing(UIMAUpdateRequestProcessorTest.java:83)
	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testMultiplierProcessing(UIMAUpdateRequestProcessorTest.java:177)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 12271 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsRecoveryZkTest
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_C3FB20DCFC9A64A6-001/init-core-data-001
   [junit4]   2> 2311330 INFO  (SUITE-HdfsRecoveryZkTest-seed#[C3FB20DCFC9A64A6]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 2311330 INFO  (SUITE-HdfsRecoveryZkTest-seed#[C3FB20DCFC9A64A6]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_C3FB20DCFC9A64A6-001/tempDir-001
   [junit4]   2> 2311331 INFO  (SUITE-HdfsRecoveryZkTest-seed#[C3FB20DCFC9A64A6]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2311331 INFO  (Thread-4605) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2311331 INFO  (Thread-4605) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2311431 INFO  (SUITE-HdfsRecoveryZkTest-seed#[C3FB20DCFC9A64A6]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:44777
   [junit4]   2> 2311440 INFO  (jetty-launcher-3321-thread-2) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 2311440 INFO  (jetty-launcher-3321-thread-1) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 2311443 INFO  (jetty-launcher-3321-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@358b5041{/solr,null,AVAILABLE}
   [junit4]   2> 2311443 INFO  (jetty-launcher-3321-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@d680027{/solr,null,AVAILABLE}
   [junit4]   2> 2311444 INFO  (jetty-launcher-3321-thread-1) [    ] o.e.j.s.ServerConnector Started ServerConnector@2589945f{HTTP/1.1,[http/1.1]}{127.0.0.1:54542}
   [junit4]   2> 2311444 INFO  (jetty-launcher-3321-thread-2) [    ] o.e.j.s.ServerConnector Started ServerConnector@1963706c{HTTP/1.1,[http/1.1]}{127.0.0.1:38068}
   [junit4]   2> 2311444 INFO  (jetty-launcher-3321-thread-1) [    ] o.e.j.s.Server Started @2315026ms
   [junit4]   2> 2311444 INFO  (jetty-launcher-3321-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=54542}
   [junit4]   2> 2311444 INFO  (jetty-launcher-3321-thread-2) [    ] o.e.j.s.Server Started @2315026ms
   [junit4]   2> 2311444 INFO  (jetty-launcher-3321-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38068}
   [junit4]   2> 2311444 ERROR (jetty-launcher-3321-thread-1) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2311445 ERROR (jetty-launcher-3321-thread-2) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2311445 INFO  (jetty-launcher-3321-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 2311445 INFO  (jetty-launcher-3321-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 2311445 INFO  (jetty-launcher-3321-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2311445 INFO  (jetty-launcher-3321-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2311445 INFO  (jetty-launcher-3321-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2311445 INFO  (jetty-launcher-3321-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2311445 INFO  (jetty-launcher-3321-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2016-11-25T19:03:25.636Z
   [junit4]   2> 2311445 INFO  (jetty-launcher-3321-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2016-11-25T19:03:25.636Z
   [junit4]   2> 2311450 INFO  (jetty-launcher-3321-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2311451 INFO  (jetty-launcher-3321-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2311459 INFO  (jetty-launcher-3321-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44777/solr
   [junit4]   2> 2311460 INFO  (jetty-launcher-3321-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44777/solr
   [junit4]   2> 2311502 INFO  (jetty-launcher-3321-thread-1) [n:127.0.0.1:54542_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:54542_solr
   [junit4]   2> 2311504 INFO  (jetty-launcher-3321-thread-2) [n:127.0.0.1:38068_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38068_solr
   [junit4]   2> 2311505 INFO  (jetty-launcher-3321-thread-1) [n:127.0.0.1:54542_solr    ] o.a.s.c.Overseer Overseer (id=96999873283620870-127.0.0.1:54542_solr-n_0000000000) starting
   [junit4]   2> 2311506 INFO  (zkCallback-3331-thread-1-processing-n:127.0.0.1:38068_solr) [n:127.0.0.1:38068_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2311506 INFO  (zkCallback-3332-thread-1-processing-n:127.0.0.1:54542_solr) [n:127.0.0.1:54542_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2311520 INFO  (jetty-launcher-3321-thread-1) [n:127.0.0.1:54542_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54542_solr
   [junit4]   2> 2311523 INFO  (zkCallback-3332-thread-1-processing-n:127.0.0.1:54542_solr) [n:127.0.0.1:54542_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2311525 INFO  (zkCallback-3331-thread-1-processing-n:127.0.0.1:38068_solr) [n:127.0.0.1:38068_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2311615 INFO  (jetty-launcher-3321-thread-1) [n:127.0.0.1:54542_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_C3FB20DCFC9A64A6-001/tempDir-001/node1/.
   [junit4]   2> 2311640 INFO  (jetty-launcher-3321-thread-2) [n:127.0.0.1:38068_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_C3FB20DCFC9A64A6-001/tempDir-001/node2/.
   [junit4]   2> 2311653 INFO  (SUITE-HdfsRecoveryZkTest-seed#[C3FB20DCFC9A64A6]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2311655 INFO  (SUITE-HdfsRecoveryZkTest-seed#[C3FB20DCFC9A64A6]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:44777/solr ready
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 2311707 WARN  (SUITE-HdfsRecoveryZkTest-seed#[C3FB20DCFC9A64A6]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 2311719 WARN  (SUITE-HdfsRecoveryZkTest-seed#[C3FB20DCFC9A64A6]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 2311722 INFO  (SUITE-HdfsRecoveryZkTest-seed#[C3FB20DCFC9A64A6]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 2311741 INFO  (SUITE-HdfsRecoveryZkTest-seed#[C3FB20DCFC9A64A6]-worker) [    ] o.m.log Extract jar:file:/export/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs to ./temp/Jetty_solaris.vm_36305_hdfs____y52ocn/webapp
   [junit4]   2> 2312289 INFO  (SUITE-HdfsRecoveryZkTest-seed#[C3FB20DCFC9A64A6]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@solaris-vm:36305
   [junit4]   2> 2312423 WARN  (SUITE-HdfsRecoveryZkTest-seed#[C3FB20DCFC9A64A6]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 2312424 INFO  (SUITE-HdfsRecoveryZkTest-seed#[C3FB20DCFC9A64A6]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 2312445 INFO  (SUITE-HdfsRecoveryZkTest-seed#[C3FB20DCFC9A64A6]-worker) [    ] o.m.log Extract jar:file:/export/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_63010_datanode____o7lro9/webapp
   [junit4]   2> 2312973 INFO  (SUITE-HdfsRecoveryZkTest-seed#[C3FB20DCFC9A64A6]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:63010
   [junit4]   2> 2313058 WARN  (SUITE-HdfsRecoveryZkTest-seed#[C3FB20DCFC9A64A6]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 2313059 INFO  (SUITE-HdfsRecoveryZkTest-seed#[C3FB20DCFC9A64A6]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 2313073 INFO  (SUITE-HdfsRecoveryZkTest-seed#[C3FB20DCFC9A64A6]-worker) [    ] o.m.log Extract jar:file:/export/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_63297_datanode____.qs1m7a/webapp
   [junit4]   2> 2313144 INFO  (IPC Server handler 3 on 65320) [    ] BlockStateChange BLOCK* processReport: from storage DS-65438158-adee-4f45-87cf-305e3bdd7b2c node DatanodeRegistration(127.0.0.1:47843, datanodeUuid=ef2b4893-c70a-4573-af81-eafda718621b, infoPort=37025, infoSecurePort=0, ipcPort=63540, storageInfo=lv=-56;cid=testClusterID;nsid=481550035;c=0), blocks: 0, hasStaleStorage: true, processing time: 1 msecs
   [junit4]   2> 2313144 INFO  (IPC Server handler 3 on 65320) [    ] BlockStateChange BLOCK* processReport: from storage DS-72137fdd-4bc8-41f0-9fd0-cb1b8deb14ce node DatanodeRegistration(127.0.0.1:47843, datanodeUuid=ef2b4893-c70a-4573-af81-eafda718621b, infoPort=37025, infoSecurePort=0, ipcPort=63540, storageInfo=lv=-56;cid=testClusterID;nsid=481550035;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 2313621 INFO  (SUITE-HdfsRecoveryZkTest-seed#[C3FB20DCFC9A64A6]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:63297
   [junit4]   2> 2313781 INFO  (IPC Server handler 4 on 65320) [    ] BlockStateChange BLOCK* processReport: from storage DS-f66dfab0-0c76-45e8-95d1-0529a30bc6ad node DatanodeRegistration(127.0.0.1:59872, datanodeUuid=7da1457e-ff96-487a-b877-b65d85f20ca9, infoPort=44396, infoSecurePort=0, ipcPort=45309, storageInfo=lv=-56;cid=testClusterID;nsid=481550035;c=0), blocks: 0, hasStaleStorage: true, processing time: 1 msecs
   [junit4]   2> 2313781 INFO  (IPC Server handler 4 on 65320) [    ] BlockStateChange BLOCK* processReport: from storage DS-5c5fc709-b99f-4b15-9831-c02ce223e575 node DatanodeRegistration(127.0.0.1:59872, datanodeUuid=7da1457e-ff96-487a-b877-b65d85f20ca9, infoPort=44396, infoSecurePort=0, ipcPort=45309, storageInfo=lv=-56;cid=testClusterID;nsid=481550035;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 2313984 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[C3FB20DCFC9A64A6]) [    ] o.a.s.SolrTestCaseJ4 ###Starting test
   [junit4]   2> 2313987 INFO  (qtp1638374409-19909) [n:127.0.0.1:54542_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf&maxShardsPerNode=1&name=recoverytest&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2313991 INFO  (OverseerThreadFactory-9576-thread-1-processing-n:127.0.0.1:54542_solr) [n:127.0.0.1:54542_solr    ] o.a.s.c.CreateCollectionCmd Create collection recoverytest
   [junit4]   2> 2314110 INFO  (qtp1638374409-19911) [n:127.0.0.1:54542_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=recoverytest_shard1_replica2&action=CREATE&numShards=1&collection=recoverytest&shard=shard1&wt=javabin&version=2
   [junit4]   2> 2314110 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=recoverytest_shard1_replica1&action=CREATE&numShards=1&collection=recoverytest&shard=shard1&wt=javabin&version=2
   [junit4]   2> 2314220 INFO  (zkCallback-3332-thread-1-processing-n:127.0.0.1:54542_solr) [n:127.0.0.1:54542_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/recoverytest/state.json] for collection [recoverytest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2314221 INFO  (zkCallback-3331-thread-1-processing-n:127.0.0.1:38068_solr) [n:127.0.0.1:38068_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/recoverytest/state.json] for collection [recoverytest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2315123 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1  x:recoverytest_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2315127 INFO  (qtp1638374409-19911) [n:127.0.0.1:54542_solr c:recoverytest s:shard1  x:recoverytest_shard1_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2315132 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1  x:recoverytest_shard1_replica1] o.a.s.s.IndexSchema [recoverytest_shard1_replica1] Schema name=minimal
   [junit4]   2> 2315134 INFO  (qtp1638374409-19911) [n:127.0.0.1:54542_solr c:recoverytest s:shard1  x:recoverytest_shard1_replica2] o.a.s.s.IndexSchema [recoverytest_shard1_replica2] Schema name=minimal
   [junit4]   2> 2315134 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1  x:recoverytest_shard1_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2315135 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1  x:recoverytest_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'recoverytest_shard1_replica1' using configuration from collection recoverytest
   [junit4]   2> 2315136 INFO  (qtp1638374409-19911) [n:127.0.0.1:54542_solr c:recoverytest s:shard1  x:recoverytest_shard1_replica2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2315137 INFO  (qtp1638374409-19911) [n:127.0.0.1:54542_solr c:recoverytest s:shard1  x:recoverytest_shard1_replica2] o.a.s.c.CoreContainer Creating SolrCore 'recoverytest_shard1_replica2' using configuration from collection recoverytest
   [junit4]   2> 2315143 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://solaris-vm:65320/data
   [junit4]   2> 2315143 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 2315144 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.SolrCore [[recoverytest_shard1_replica1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_C3FB20DCFC9A64A6-001/tempDir-001/node2/recoverytest_shard1_replica1], dataDir=[hdfs://solaris-vm:65320/data/recoverytest/core_node1/data/]
   [junit4]   2> 2315144 INFO  (qtp1638374409-19911) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://solaris-vm:65320/data
   [junit4]   2> 2315144 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://solaris-vm:65320/data/recoverytest/core_node1/data/snapshot_metadata
   [junit4]   2> 2315144 INFO  (qtp1638374409-19911) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 2315144 INFO  (qtp1638374409-19911) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.SolrCore [[recoverytest_shard1_replica2] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_C3FB20DCFC9A64A6-001/tempDir-001/node1/recoverytest_shard1_replica2], dataDir=[hdfs://solaris-vm:65320/data/recoverytest/core_node2/data/]
   [junit4]   2> 2315144 INFO  (qtp1638374409-19911) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://solaris-vm:65320/data/recoverytest/core_node2/data/snapshot_metadata
   [junit4]   2> 2315171 INFO  (qtp1638374409-19911) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 2315171 INFO  (qtp1638374409-19911) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
   [junit4]   2> 2315171 INFO  (qtp1638374409-19911) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 2315171 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 2315171 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
   [junit4]   2> 2315171 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 2315252 INFO  (qtp1638374409-19911) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 2315255 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 2315255 INFO  (qtp1638374409-19911) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://solaris-vm:65320/data/recoverytest/core_node2/data
   [junit4]   2> 2315256 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://solaris-vm:65320/data/recoverytest/core_node1/data
   [junit4]   2> 2315297 INFO  (qtp1638374409-19911) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://solaris-vm:65320/data/recoverytest/core_node2/data/index
   [junit4]   2> 2315304 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://solaris-vm:65320/data/recoverytest/core_node1/data/index
   [junit4]   2> 2315310 INFO  (qtp1638374409-19911) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 2315310 INFO  (qtp1638374409-19911) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
   [junit4]   2> 2315310 INFO  (qtp1638374409-19911) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 2315316 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 2315316 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
   [junit4]   2> 2315316 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 2315408 INFO  (qtp1638374409-19911) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 2315413 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 2315627 INFO  (IPC Server handler 6 on 65320) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:59872 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-65438158-adee-4f45-87cf-305e3bdd7b2c:NORMAL:127.0.0.1:47843|RBW], ReplicaUC[[DISK]DS-5c5fc709-b99f-4b15-9831-c02ce223e575:NORMAL:127.0.0.1:59872|RBW]]} size 0
   [junit4]   2> 2315630 INFO  (IPC Server handler 5 on 65320) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:59872 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-f66dfab0-0c76-45e8-95d1-0529a30bc6ad:NORMAL:127.0.0.1:59872|RBW], ReplicaUC[[DISK]DS-72137fdd-4bc8-41f0-9fd0-cb1b8deb14ce:NORMAL:127.0.0.1:47843|RBW]]} size 0
   [junit4]   2> 2315630 INFO  (IPC Server handler 7 on 65320) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:47843 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-65438158-adee-4f45-87cf-305e3bdd7b2c:NORMAL:127.0.0.1:47843|RBW], ReplicaUC[[DISK]DS-5c5fc709-b99f-4b15-9831-c02ce223e575:NORMAL:127.0.0.1:59872|RBW]]} size 0
   [junit4]   2> 2315630 INFO  (IPC Server handler 7 on 65320) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:47843 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-f66dfab0-0c76-45e8-95d1-0529a30bc6ad:NORMAL:127.0.0.1:59872|RBW], ReplicaUC[[DISK]DS-72137fdd-4bc8-41f0-9fd0-cb1b8deb14ce:NORMAL:127.0.0.1:47843|RBW]]} size 0
   [junit4]   2> 2315664 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 2315664 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2315664 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
   [junit4]   2> 2315696 INFO  (qtp1638374409-19911) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 2315696 INFO  (qtp1638374409-19911) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2315696 INFO  (qtp1638374409-19911) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
   [junit4]   2> 2315709 INFO  (qtp1638374409-19911) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2315709 INFO  (qtp1638374409-19911) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2315709 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2315710 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2315731 INFO  (qtp1638374409-19911) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@4e4b38c5[recoverytest_shard1_replica2] main]
   [junit4]   2> 2315732 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1630bbde[recoverytest_shard1_replica1] main]
   [junit4]   2> 2315733 INFO  (qtp1638374409-19911) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 2315733 INFO  (qtp1638374409-19911) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 2315734 INFO  (qtp1638374409-19911) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2315734 INFO  (searcherExecutor-9582-thread-1-processing-n:127.0.0.1:54542_solr x:recoverytest_shard1_replica2 s:shard1 c:recoverytest r:core_node2) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.SolrCore [recoverytest_shard1_replica2] Registered new searcher Searcher@4e4b38c5[recoverytest_shard1_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2315734 INFO  (qtp1638374409-19911) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1551997977152716800
   [junit4]   2> 2315734 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 2315735 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 2315735 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2315736 INFO  (searcherExecutor-9581-thread-1-processing-n:127.0.0.1:38068_solr x:recoverytest_shard1_replica1 s:shard1 c:recoverytest r:core_node1) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.SolrCore [recoverytest_shard1_replica1] Registered new searcher Searcher@1630bbde[recoverytest_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2315736 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1551997977154813952
   [junit4]   2> 2315749 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2315749 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2315749 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/
   [junit4]   2> 2315750 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.PeerSync PeerSync: core=recoverytest_shard1_replica1 url=http://127.0.0.1:38068/solr START replicas=[http://127.0.0.1:54542/solr/recoverytest_shard1_replica2/] nUpdates=100
   [junit4]   2> 2315751 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.PeerSync PeerSync: core=recoverytest_shard1_replica1 url=http://127.0.0.1:38068/solr DONE.  We have no versions.  sync failed.
   [junit4]   2> 2315751 INFO  (zkCallback-3331-thread-1-processing-n:127.0.0.1:38068_solr) [n:127.0.0.1:38068_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/recoverytest/state.json] for collection [recoverytest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2315751 INFO  (zkCallback-3332-thread-2-processing-n:127.0.0.1:54542_solr) [n:127.0.0.1:54542_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/recoverytest/state.json] for collection [recoverytest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2315755 INFO  (qtp1638374409-19914) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.c.S.Request [recoverytest_shard1_replica2]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 2315756 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 2315756 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 2315763 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/ shard1
   [junit4]   2> 2315868 INFO  (zkCallback-3332-thread-2-processing-n:127.0.0.1:54542_solr) [n:127.0.0.1:54542_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/recoverytest/state.json] for collection [recoverytest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2315868 INFO  (zkCallback-3331-thread-1-processing-n:127.0.0.1:38068_solr) [n:127.0.0.1:38068_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/recoverytest/state.json] for collection [recoverytest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2315870 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2315873 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=recoverytest_shard1_replica1&action=CREATE&numShards=1&collection=recoverytest&shard=shard1&wt=javabin&version=2} status=0 QTime=1763
   [junit4]   2> 2315976 INFO  (zkCallback-3331-thread-1-processing-n:127.0.0.1:38068_solr) [n:127.0.0.1:38068_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/recoverytest/state.json] for collection [recoverytest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2315976 INFO  (zkCallback-3332-thread-2-processing-n:127.0.0.1:54542_solr) [n:127.0.0.1:54542_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/recoverytest/state.json] for collection [recoverytest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2316753 INFO  (qtp1638374409-19911) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=recoverytest_shard1_replica2&action=CREATE&numShards=1&collection=recoverytest&shard=shard1&wt=javabin&version=2} status=0 QTime=2643
   [junit4]   2> 2316759 INFO  (qtp1638374409-19909) [n:127.0.0.1:54542_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 2316858 INFO  (zkCallback-3332-thread-2-processing-n:127.0.0.1:54542_solr) [n:127.0.0.1:54542_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/recoverytest/state.json] for collection [recoverytest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2316860 INFO  (zkCallback-3331-thread-1-processing-n:127.0.0.1:38068_solr) [n:127.0.0.1:38068_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/recoverytest/state.json] for collection [recoverytest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2317760 INFO  (qtp1638374409-19909) [n:127.0.0.1:54542_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=2&collection.configName=conf&maxShardsPerNode=1&name=recoverytest&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=3774
   [junit4]   2> 2317761 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[C3FB20DCFC9A64A6]) [    ] o.a.s.c.RecoveryZkTest Indexing 1350 documents
   [junit4]   2> 2317788 DEBUG (qtp237906199-19917) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.HdfsTransactionLog Opening new tlog hdfs tlog{file=hdfs://solaris-vm:65320/data/recoverytest/core_node1/data/tlog/tlog.0000000000000000000 refcount=1}
   [junit4]   2> 2317800 DEBUG (qtp1638374409-19915) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.HdfsTransactionLog Opening new tlog hdfs tlog{file=hdfs://solaris-vm:65320/data/recoverytest/core_node2/data/tlog/tlog.0000000000000000000 refcount=1}
   [junit4]   2> 2317820 INFO  (qtp1638374409-19915) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{add=[2-0 (1551997979284471809)]} 0 30
   [junit4]   2> 2317820 INFO  (qtp237906199-19919) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-0 (1551997979284471809)]} 0 53
   [junit4]   2> 2317824 INFO  (qtp1638374409-19910) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{add=[1-0 (1551997979284471808)]} 0 33
   [junit4]   2> 2317824 INFO  (qtp1638374409-19913) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{add=[2-1 (1551997979341094912)]} 0 1
   [junit4]   2> 2317824 INFO  (qtp237906199-19917) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-0 (1551997979284471808)]} 0 57
   [junit4]   2> 2317824 INFO  (qtp237906199-19920) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-1 (1551997979341094912)]} 0 3
   [junit4]   2> 2317829 INFO  (qtp1638374409-19911) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{add=[1-1 (1551997979345289217)]} 0 3
   [junit4]   2> 2317830 INFO  (qtp1638374409-19914) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{add=[2-2 (1551997979345289216)]} 0 4
   [junit4]   2> 2317830 INFO  (qtp237906199-19905) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-2 (1551997979345289216)]} 0 5
   [junit4]   2> 2317831 INFO  (qtp237906199-19921) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-1 (1551997979345289217)]} 0 5
   [junit4]   2> 2317835 INFO  (qtp1638374409-19907) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{add=[1-2 (1551997979352629249)]} 0 2
   [junit4]   2> 2317835 INFO  (qtp1638374409-19909) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{add=[2-3 (1551997979352629248)]} 0 2
   [junit4]   2> 2317836 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-3 (1551997979352629248)]} 0 3
   [junit4]   2> 2317837 INFO  (qtp237906199-19918) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-2 (1551997979352629249)]} 0 5
   [junit4]   2> 2317840 INFO  (qtp1638374409-19915) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{add=[2-4 (1551997979357872128)]} 0 3
   [junit4]   2> 2317841 INFO  (qtp237906199-19919) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-4 (1551997979357872128)]} 0 4
   [junit4]   2> 2317842 INFO  (qtp1638374409-19910) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{add=[1-3 (1551997979361017856)]} 0 1
   [junit4]   2> 2317842 INFO  (qtp237906199-19917) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-3 (1551997979361017856)]} 0 2
   [junit4]   2> 2317844 INFO  (qtp1638374409-19913) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{add=[2-5 (1551997979363115008)]} 0 1
   [junit4]   2> 2317844 INFO  (qtp237906199-19920) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-5 (1551997979363115008)]} 0 2
   [junit4]   2> 2317845 INFO  (qtp1638374409-19910) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{add=[1-4 (1551997979364163584)]} 0 1
   [junit4]   2> 2317845 INFO  (qtp237906199-19905) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-4 (1551997979364163584)]} 0 2
   [junit4]   2> 2317847 INFO  (qtp1638374409-19914) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{add=[2-6 (1551997979366260736)]} 0 1
   [junit4]   2> 2317848 INFO  (qtp237906199-19921) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-6 (1551997979366260736)]} 0 3
   [junit4]   2> 2317850 INFO  (qtp1638374409-19907) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{add=[1-5 (1551997979367309312)]} 0 4
   [junit4]   2> 2317852 INFO  (qtp1638374409-19914) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{delete=[2-1 (-1551997979370455040)]} 0 2
   [junit4]   2> 2317852 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-5 (1551997979367309312)]} 0 6
   [junit4]   2> 2317853 INFO  (qtp237906199-19918) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[2-1 (-1551997979370455040)]} 0 3
   [junit4]   2> 2317855 INFO  (qtp1638374409-19907) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{add=[2-7 (1551997979375697920)]} 0 1
   [junit4]   2> 2317856 INFO  (qtp237906199-19919) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-7 (1551997979375697920)]} 0 2
   [junit4]   2> 2317858 INFO  (qtp1638374409-19907) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{add=[2-8 (1551997979377795072)]} 0 1
   [junit4]   2> 2317858 INFO  (qtp237906199-19919) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-8 (1551997979377795072)]} 0 2
   [junit4]   2> 2317861 INFO  (qtp1638374409-19907) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{add=[2-9 (1551997979380940800)]} 0 1
   [junit4]   2> 2317861 INFO  (qtp237906199-19919) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-9 (1551997979380940800)]} 0 2
   [junit4]   2> 2317863 INFO  (qtp1638374409-19907) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{add=[1-6 (1551997979383037952)]} 0 1
   [junit4]   2> 2317864 INFO  (qtp237906199-19919) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-6 (1551997979383037952)]} 0 2
   [junit4]   2> 2317864 INFO  (qtp1638374409-19913) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{delete=[2-3 (-1551997979384086528)]} 0 1
   [junit4]   2> 2317865 INFO  (qtp237906199-19917) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[2-3 (-1551997979384086528)]} 0 2
   [junit4]   2> 2317866 INFO  (qtp1638374409-19910) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{delete=[1-1 (-1551997979386183680)]} 0 1
   [junit4]   2> 2317866 INFO  (qtp237906199-19920) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-1 (-1551997979386183680)]} 0 1
   [junit4]   2> 2317867 INFO  (qtp1638374409-19909) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{add=[2-10 (1551997979387232256)]} 0 1
   [junit4]   2> 2317867 INFO  (qtp237906199-19905) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-10 (1551997979387232256)]} 0 2
   [junit4]   2> 2317868 INFO  (qtp1638374409-19910) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{add=[1-7 (1551997979389329408)]} 0 1
   [junit4]   2> 2317870 INFO  (qtp237906199-19921) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-7 (1551997979389329408)]} 0 2
   [junit4]   2> 2317870 INFO  (qtp1638374409-19909) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{add=[2-11 (1551997979390377984)]} 0 1
   [junit4]   2> 2317870 INFO  (qtp237906199-19912) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-11 (1551997979390377984)]} 0 2
   [junit4]   2> 2317872 INFO  (qtp1638374409-19910) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{add=[1-8 (1551997979393523712)]} 0 0
   [junit4]   2> 2317872 INFO  (qtp237906199-19918) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-8 (1551997979393523712)]} 0 1
   [junit4]   2> 2317874 INFO  (qtp1638374409-19910) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{add=[1-9 (1551997979395620864)]} 0 0
   [junit4]   2> 2317874 INFO  (qtp237906199-19918) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-9 (1551997979395620864)]} 0 1
   [junit4]   2> 2317876 INFO  (qtp1638374409-19910) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{delete=[1-3 (-1551997979396669440)]} 0 1
   [junit4]   2> 2317876 INFO  (qtp237906199-19918) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-3 (-1551997979396669440)]} 0 1
   [junit4]   2> 2317878 INFO  (qtp1638374409-19910) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{add=[1-10 (1551997979399815168)]} 0 0
   [junit4]   2> 2317878 INFO  (qtp237906199-19918) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-10 (1551997979399815168)]} 0 1
   [junit4]   2> 2317880 INFO  (qtp1638374409-19910) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{add=[1-11 (1551997979401912320)]} 0 1
   [junit4]   2> 2317880 INFO  (qtp237906199-19918) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-11 (1551997979401912320)]} 0 1
   [junit4]   2> 2317882 INFO  (qtp1638374409-19910) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{add=[1-12 (1551997979404009472)]} 0 0
   [junit4]   2> 2317882 INFO  (qtp237906199-19918) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-12 (1551997979404009472)]} 0 1
   [junit4]   2> 2317884 INFO  (qtp1638374409-19910) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{delete=[1-4 (-1551997979406106624)]} 0 0
   [junit4]   2> 2317884 INFO  (qtp237906199-19918) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-4 (-1551997979406106624)]} 0 1
   [junit4]   2> 2317886 INFO  (qtp1638374409-19910) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{add=[1-13 (1551997979408203776)]} 0 0
   [junit4]   2> 2317886 INFO  (qtp237906199-19918) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-13 (1551997979408203776)]} 0 1
   [junit4]   2> 2317888 INFO  (qtp1638374409-19910) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{delete=[1-6 (-1551997979410300928)]} 0 0
   [junit4]   2> 2317889 INFO  (qtp237906199-19918) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-6 (-1551997979410300928)]} 0 1
   [junit4]   2> 2317890 INFO  (qtp1638374409-19910) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{add=[1-14 (1551997979412398080)]} 0 0
   [junit4]   2> 2317891 INFO  (qtp237906199-19918) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-14 (1551997979412398080)]} 0 1
   [junit4]   2> 2317892 INFO  (qtp1638374409-19910) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{add=[1-15 (1551997979414495232)]} 0 0
   [junit4]   2> 2317892 INFO  (qtp237906199-19918) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-15 (1551997979414495232)]} 0 1
   [junit4]   2> 2317894 INFO  (qtp1638374409-19910) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{add=[1-16 (1551997979416592384)]} 0 0
   [junit4]   2> 2317895 INFO  (qtp237906199-19918) [n:127.0.0.1:38068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-16 (1551997979416592384)]} 0 1
   [junit4]   2> 2317896 INFO  (qtp1638374409-19910) [n:127.0.0.1:54542_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38068/solr/recoverytest_shard1_replica1/&wt=javabin&version=2}{delete=[1-7 (-1551997979418689536)]} 0 0
   [junit4]   2> 2317

[...truncated too long message...]

Tracker.java:43)
   [junit4]    > 	at org.apache.solr.update.HdfsTransactionLog.<init>(HdfsTransactionLog.java:130)
   [junit4]    > 	at org.apache.solr.update.HdfsUpdateLog.init(HdfsUpdateLog.java:202)
   [junit4]    > 	at org.apache.solr.update.UpdateHandler.<init>(UpdateHandler.java:137)
   [junit4]    > 	at org.apache.solr.update.UpdateHandler.<init>(UpdateHandler.java:94)
   [junit4]    > 	at org.apache.solr.update.DirectUpdateHandler2.<init>(DirectUpdateHandler2.java:102)
   [junit4]    > 	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
   [junit4]    > 	at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:723)
   [junit4]    > 	at org.apache.solr.core.SolrCore.createUpdateHandler(SolrCore.java:785)
   [junit4]    > 	at org.apache.solr.core.SolrCore.initUpdateHandler(SolrCore.java:1024)
   [junit4]    > 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:889)
   [junit4]    > 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:793)
   [junit4]    > 	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:868)
   [junit4]    > 	at org.apache.solr.core.CoreContainer.lambda$load$0(CoreContainer.java:517)
   [junit4]    > 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]    > 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([C3FB20DCFC9A64A6]:0)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:266)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4] Completed [518/657 (1!)] on J0 in 49.95s, 1 test, 1 failure <<< FAILURES!

[...truncated 11547 lines...]
   [junit4] Suite: org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/contrib/solr-uima/test/J0/temp/solr.uima.processor.UIMAUpdateRequestProcessorTest_A4236383C33B069F-001/init-core-data-001
   [junit4]   2> log4j:WARN No appenders could be found for logger (org.eclipse.jetty.util.log).
   [junit4]   2> log4j:WARN Please initialize the log4j system properly.
   [junit4]   2> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
   [junit4]   2> Nov 25, 2016 10:19:57 PM WhitespaceTokenizer initialize
   [junit4]   2> INFO: "Whitespace tokenizer successfully initialized"
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=UIMAUpdateRequestProcessorTest -Dtests.method=testTwoUpdates -Dtests.seed=A4236383C33B069F -Dtests.slow=true -Dtests.locale=en-ZA -Dtests.timezone=Indian/Comoro -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   6.23s J0 | UIMAUpdateRequestProcessorTest.testTwoUpdates <<<
   [junit4]    > Throwable #1: org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">1</field><field name="text">The Apache Software Foundation is happy to announce BarCampApache Sydney, Australia, the first ASF-backed event in the Southern Hemisphere!</field></doc></add>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([A4236383C33B069F:A96934DCDB146431]:0)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
   [junit4]    > 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testTwoUpdates(UIMAUpdateRequestProcessorTest.java:101)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=UIMAUpdateRequestProcessorTest -Dtests.method=testErrorHandling -Dtests.seed=A4236383C33B069F -Dtests.slow=true -Dtests.locale=en-ZA -Dtests.timezone=Indian/Comoro -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.04s J0 | UIMAUpdateRequestProcessorTest.testErrorHandling <<<
   [junit4]    > Throwable #1: org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
   [junit4]    >   Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([A4236383C33B069F:91D71DBD95ED874B]:0)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
   [junit4]    > 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testErrorHandling(UIMAUpdateRequestProcessorTest.java:137)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=UIMAUpdateRequestProcessorTest -Dtests.method=testProcessing -Dtests.seed=A4236383C33B069F -Dtests.slow=true -Dtests.locale=en-ZA -Dtests.timezone=Indian/Comoro -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.02s J0 | UIMAUpdateRequestProcessorTest.testProcessing <<<
   [junit4]    > Throwable #1: org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
   [junit4]    >   Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([A4236383C33B069F:B7AD67F184C88735]:0)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
   [junit4]    > 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing(UIMAUpdateRequestProcessorTest.java:83)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=UIMAUpdateRequestProcessorTest -Dtests.method=testMultiplierProcessing -Dtests.seed=A4236383C33B069F -Dtests.slow=true -Dtests.locale=en-ZA -Dtests.timezone=Indian/Comoro -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.01s J0 | UIMAUpdateRequestProcessorTest.testMultiplierProcessing <<<
   [junit4]    > Throwable #1: org.apache.solr.common.SolrException: unknown UpdateRequestProcessorChain: <add><doc><field name="id">2312312321312</field><field name="text">SpellCheckComponent got improvement related to recent Lucene changes. 
   [junit4]    >   Add support for specifying Spelling SuggestWord Comparator to Lucene spell checkers for SpellCheckComponent. Issue SOLR-2053 is already fixed, patch is attached if you need it, but it is also committed to trunk and 3_x branch. Last Lucene European Conference has been held in Prague.</field></doc></add>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([A4236383C33B069F:9F22A92EC46F2A6B]:0)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessingChain(SolrCore.java:1214)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1222)
   [junit4]    > 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:52)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.addDoc(SolrTestCaseJ4.java:1031)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testProcessing(UIMAUpdateRequestProcessorTest.java:83)
   [junit4]    > 	at org.apache.solr.uima.processor.UIMAUpdateRequestProcessorTest.testMultiplierProcessing(UIMAUpdateRequestProcessorTest.java:177)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Nov 25, 2016 10:20:01 PM WhitespaceTokenizer initialize
   [junit4]   2> INFO: "Whitespace tokenizer successfully initialized"
   [junit4]   2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/contrib/solr-uima/test/J0/temp/solr.uima.processor.UIMAUpdateRequestProcessorTest_A4236383C33B069F-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=470, maxMBSortInHeap=6.810746772647052, sim=RandomSimilarity(queryNorm=true): {}, locale=en-ZA, timezone=Indian/Comoro
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=3,threads=1,free=123138888,total=367001600
   [junit4]   2> NOTE: All tests run in this JVM: [UIMAUpdateRequestProcessorTest]
   [junit4] Completed [2/2 (1!)] on J0 in 16.81s, 6 tests, 4 errors <<< FAILURES!

[...truncated 51249 lines...]