You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/05/01 13:42:35 UTC

[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.7.0_60-ea-b14) - Build # 10206 - Failure!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10206/
Java: 64bit/jdk1.7.0_60-ea-b14 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

3 tests failed.
REGRESSION:  org.apache.lucene.analysis.core.TestAllAnalyzersHaveFactories.test

Error Message:
A SPI class of type org.apache.lucene.analysis.util.TokenFilterFactory with name 'Cranky' does not exist. You need to add the corresponding JAR file supporting this SPI to your classpath. The current classpath supports the following names: [apostrophe, arabicnormalization, arabicstem, bulgarianstem, brazilianstem, cjkbigram, cjkwidth, soraninormalization, soranistem, commongrams, commongramsquery, dictionarycompoundword, hyphenationcompoundword, lowercase, stop, type, uppercase, czechstem, germanlightstem, germanminimalstem, germannormalization, germanstem, greeklowercase, greekstem, englishminimalstem, englishpossessive, kstem, porterstem, spanishlightstem, persiannormalization, finnishlightstem, frenchlightstem, frenchminimalstem, irishlowercase, galicianminimalstem, galicianstem, hindinormalization, hindistem, hungarianlightstem, hunspellstem, indonesianstem, indicnormalization, italianlightstem, latvianstem, asciifolding, capitalization, codepointcount, hyphenatedwords, keepword, keywordmarker, keywordrepeat, length, limittokencount, limittokenposition, removeduplicates, stemmeroverride, trim, truncate, worddelimiter, scandinavianfolding, scandinaviannormalization, edgengram, ngram, norwegianlightstem, norwegianminimalstem, patternreplace, patterncapturegroup, delimitedpayload, numericpayload, tokenoffsetpayload, typeaspayload, portugueselightstem, portugueseminimalstem, portuguesestem, reversestring, russianlightstem, shingle, snowballporter, classic, standard, swedishlightstem, synonym, thaiword, turkishlowercase, elision]

Stack Trace:
java.lang.IllegalArgumentException: A SPI class of type org.apache.lucene.analysis.util.TokenFilterFactory with name 'Cranky' does not exist. You need to add the corresponding JAR file supporting this SPI to your classpath. The current classpath supports the following names: [apostrophe, arabicnormalization, arabicstem, bulgarianstem, brazilianstem, cjkbigram, cjkwidth, soraninormalization, soranistem, commongrams, commongramsquery, dictionarycompoundword, hyphenationcompoundword, lowercase, stop, type, uppercase, czechstem, germanlightstem, germanminimalstem, germannormalization, germanstem, greeklowercase, greekstem, englishminimalstem, englishpossessive, kstem, porterstem, spanishlightstem, persiannormalization, finnishlightstem, frenchlightstem, frenchminimalstem, irishlowercase, galicianminimalstem, galicianstem, hindinormalization, hindistem, hungarianlightstem, hunspellstem, indonesianstem, indicnormalization, italianlightstem, latvianstem, asciifolding, capitalization, codepointcount, hyphenatedwords, keepword, keywordmarker, keywordrepeat, length, limittokencount, limittokenposition, removeduplicates, stemmeroverride, trim, truncate, worddelimiter, scandinavianfolding, scandinaviannormalization, edgengram, ngram, norwegianlightstem, norwegianminimalstem, patternreplace, patterncapturegroup, delimitedpayload, numericpayload, tokenoffsetpayload, typeaspayload, portugueselightstem, portugueseminimalstem, portuguesestem, reversestring, russianlightstem, shingle, snowballporter, classic, standard, swedishlightstem, synonym, thaiword, turkishlowercase, elision]
	at __randomizedtesting.SeedInfo.seed([358F86B7E9419656:BDDBB96D47BDFBAE]:0)
	at org.apache.lucene.analysis.util.AnalysisSPILoader.lookupClass(AnalysisSPILoader.java:122)
	at org.apache.lucene.analysis.util.TokenFilterFactory.lookupClass(TokenFilterFactory.java:42)
	at org.apache.lucene.analysis.core.TestAllAnalyzersHaveFactories.test(TestAllAnalyzersHaveFactories.java:148)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
	at java.lang.Thread.run(Thread.java:744)


REGRESSION:  org.apache.lucene.analysis.core.TestRandomChains.testRandomChainsWithLargeStrings

Error Message:
TokenStream implementation classes or at least their incrementToken() implementation must be final

Stack Trace:
java.lang.AssertionError: TokenStream implementation classes or at least their incrementToken() implementation must be final
	at __randomizedtesting.SeedInfo.seed([358F86B7E9419656:5FD439A6B00FB6A5]:0)
	at org.apache.lucene.analysis.TokenStream.assertFinal(TokenStream.java:120)
	at org.apache.lucene.analysis.TokenStream.<init>(TokenStream.java:101)
	at org.apache.lucene.analysis.TokenFilter.<init>(TokenFilter.java:33)
	at org.apache.lucene.analysis.CrankyTokenFilter.<init>(CrankyTokenFilter.java:35)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at org.apache.lucene.analysis.core.TestRandomChains$MockRandomAnalyzer.createComponent(TestRandomChains.java:681)
	at org.apache.lucene.analysis.core.TestRandomChains$MockRandomAnalyzer.newFilterChain(TestRandomChains.java:793)
	at org.apache.lucene.analysis.core.TestRandomChains$MockRandomAnalyzer.toString(TestRandomChains.java:670)
	at java.lang.String.valueOf(String.java:2847)
	at java.lang.StringBuilder.append(StringBuilder.java:128)
	at org.apache.lucene.analysis.core.TestRandomChains.testRandomChainsWithLargeStrings(TestRandomChains.java:923)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
	at java.lang.Thread.run(Thread.java:744)


REGRESSION:  org.apache.lucene.analysis.core.TestRandomChains.testRandomChains

Error Message:
TokenStream implementation classes or at least their incrementToken() implementation must be final

Stack Trace:
java.lang.AssertionError: TokenStream implementation classes or at least their incrementToken() implementation must be final
	at __randomizedtesting.SeedInfo.seed([358F86B7E9419656:86EAFD6AE538B96]:0)
	at org.apache.lucene.analysis.TokenStream.assertFinal(TokenStream.java:120)
	at org.apache.lucene.analysis.TokenStream.<init>(TokenStream.java:101)
	at org.apache.lucene.analysis.TokenFilter.<init>(TokenFilter.java:33)
	at org.apache.lucene.analysis.CrankyTokenFilter.<init>(CrankyTokenFilter.java:35)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at org.apache.lucene.analysis.core.TestRandomChains$MockRandomAnalyzer.createComponent(TestRandomChains.java:681)
	at org.apache.lucene.analysis.core.TestRandomChains$MockRandomAnalyzer.newFilterChain(TestRandomChains.java:793)
	at org.apache.lucene.analysis.core.TestRandomChains$MockRandomAnalyzer.toString(TestRandomChains.java:670)
	at java.lang.String.valueOf(String.java:2847)
	at java.lang.StringBuilder.append(StringBuilder.java:128)
	at org.apache.lucene.analysis.core.TestRandomChains.testRandomChains(TestRandomChains.java:904)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
	at java.lang.Thread.run(Thread.java:744)




Build Log:
[...truncated 5568 lines...]
   [junit4] Suite: org.apache.lucene.analysis.core.TestRandomChains
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestRandomChains -Dtests.method=testRandomChainsWithLargeStrings -Dtests.seed=358F86B7E9419656 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ms_MY -Dtests.timezone=Asia/Kuala_Lumpur -Dtests.file.encoding=ISO-8859-1
   [junit4] FAILURE 6.12s J1 | TestRandomChains.testRandomChainsWithLargeStrings <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: TokenStream implementation classes or at least their incrementToken() implementation must be final
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([358F86B7E9419656:5FD439A6B00FB6A5]:0)
   [junit4]    > 	at org.apache.lucene.analysis.TokenStream.assertFinal(TokenStream.java:120)
   [junit4]    > 	at org.apache.lucene.analysis.TokenStream.<init>(TokenStream.java:101)
   [junit4]    > 	at org.apache.lucene.analysis.TokenFilter.<init>(TokenFilter.java:33)
   [junit4]    > 	at org.apache.lucene.analysis.CrankyTokenFilter.<init>(CrankyTokenFilter.java:35)
   [junit4]    > 	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
   [junit4]    > 	at org.apache.lucene.analysis.core.TestRandomChains$MockRandomAnalyzer.createComponent(TestRandomChains.java:681)
   [junit4]    > 	at org.apache.lucene.analysis.core.TestRandomChains$MockRandomAnalyzer.newFilterChain(TestRandomChains.java:793)
   [junit4]    > 	at org.apache.lucene.analysis.core.TestRandomChains$MockRandomAnalyzer.toString(TestRandomChains.java:670)
   [junit4]    > 	at java.lang.String.valueOf(String.java:2847)
   [junit4]    > 	at java.lang.StringBuilder.append(StringBuilder.java:128)
   [junit4]    > 	at org.apache.lucene.analysis.core.TestRandomChains.testRandomChainsWithLargeStrings(TestRandomChains.java:923)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestRandomChains -Dtests.method=testRandomChains -Dtests.seed=358F86B7E9419656 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ms_MY -Dtests.timezone=Asia/Kuala_Lumpur -Dtests.file.encoding=ISO-8859-1
   [junit4] FAILURE 15.3s J1 | TestRandomChains.testRandomChains <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: TokenStream implementation classes or at least their incrementToken() implementation must be final
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([358F86B7E9419656:86EAFD6AE538B96]:0)
   [junit4]    > 	at org.apache.lucene.analysis.TokenStream.assertFinal(TokenStream.java:120)
   [junit4]    > 	at org.apache.lucene.analysis.TokenStream.<init>(TokenStream.java:101)
   [junit4]    > 	at org.apache.lucene.analysis.TokenFilter.<init>(TokenFilter.java:33)
   [junit4]    > 	at org.apache.lucene.analysis.CrankyTokenFilter.<init>(CrankyTokenFilter.java:35)
   [junit4]    > 	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
   [junit4]    > 	at org.apache.lucene.analysis.core.TestRandomChains$MockRandomAnalyzer.createComponent(TestRandomChains.java:681)
   [junit4]    > 	at org.apache.lucene.analysis.core.TestRandomChains$MockRandomAnalyzer.newFilterChain(TestRandomChains.java:793)
   [junit4]    > 	at org.apache.lucene.analysis.core.TestRandomChains$MockRandomAnalyzer.toString(TestRandomChains.java:670)
   [junit4]    > 	at java.lang.String.valueOf(String.java:2847)
   [junit4]    > 	at java.lang.StringBuilder.append(StringBuilder.java:128)
   [junit4]    > 	at org.apache.lucene.analysis.core.TestRandomChains.testRandomChains(TestRandomChains.java:904)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/analysis/common/test/J1/./lucene.analysis.core.TestRandomChains-358F86B7E9419656-001
   [junit4]   2> NOTE: test params are: codec=Asserting, sim=DefaultSimilarity, locale=ms_MY, timezone=Asia/Kuala_Lumpur
   [junit4]   2> NOTE: Linux 3.8.0-38-generic amd64/Oracle Corporation 1.7.0_60-ea (64-bit)/cpus=8,threads=1,free=75951208,total=126418944
   [junit4]   2> NOTE: All tests run in this JVM: [TestBulgarianStemFilterFactory, TestSnowballPorterFilterFactory, TestTrimFilterFactory, TestHungarianLightStemFilterFactory, TestArmenianAnalyzer, TestEnglishMinimalStemFilterFactory, DateRecognizerSinkTokenizerTest, TokenOffsetPayloadTokenFilterTest, TestKStemmer, QueryAutoStopWordAnalyzerTest, TestSpanishLightStemFilterFactory, EdgeNGramTokenizerTest, DelimitedPayloadTokenFilterTest, TestBulgarianStemmer, TestCompoundWordTokenFilter, TestTurkishAnalyzer, TestTruncateTokenFilterFactory, TestPortugueseAnalyzer, TestPortugueseStemFilter, TestComplexPrefix, TestDanishAnalyzer, TestCJKBigramFilterFactory, TestLatvianStemFilterFactory, TestIndicNormalizer, TestPerFieldAnalyzerWrapper, TestNorwegianMinimalStemFilter, TestHunspellStemFilterFactory, TestCJKWidthFilter, TestCommonGramsFilterFactory, TestCommonGramsQueryFilterFactory, TestCodepointCountFilter, TestHindiFilters, TestClassicAnalyzer, TestFrenchMinimalStemFilterFactory, TestFinnishAnalyzer, NGramTokenFilterTest, TestSegmentingTokenizerBase, TestCollationKeyAnalyzer, TestGalicianStemFilterFactory, TestEnglishMinimalStemFilter, TestHindiAnalyzer, CommonGramsFilterTest, TestMorph, TestGalicianAnalyzer, TestWordnetSynonymParser, TestStandardAnalyzer, TestPortugueseStemFilterFactory, TestArabicFilters, TestStemmerOverrideFilterFactory, TestGreekStemmer, TestFrenchAnalyzer, TestRollingCharBuffer, TestCharArrayMap, TestDictionaryCompoundWordTokenFilterFactory, TestSpanishLightStemFilter, TestIrishLowerCaseFilter, TestCharacterUtils, TestCzechStemmer, TestCapitalizationFilter, TestStemmer, TestUAX29URLEmailTokenizer, TestGreekStemFilterFactory, TestSnowball, TestRandomChains]
   [junit4] Completed on J1 in 21.83s, 2 tests, 2 failures <<< FAILURES!

[...truncated 109 lines...]
   [junit4] Suite: org.apache.lucene.analysis.core.TestAllAnalyzersHaveFactories
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestAllAnalyzersHaveFactories -Dtests.method=test -Dtests.seed=358F86B7E9419656 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_SV -Dtests.timezone=America/Yellowknife -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.19s J1 | TestAllAnalyzersHaveFactories.test <<<
   [junit4]    > Throwable #1: java.lang.IllegalArgumentException: A SPI class of type org.apache.lucene.analysis.util.TokenFilterFactory with name 'Cranky' does not exist. You need to add the corresponding JAR file supporting this SPI to your classpath. The current classpath supports the following names: [apostrophe, arabicnormalization, arabicstem, bulgarianstem, brazilianstem, cjkbigram, cjkwidth, soraninormalization, soranistem, commongrams, commongramsquery, dictionarycompoundword, hyphenationcompoundword, lowercase, stop, type, uppercase, czechstem, germanlightstem, germanminimalstem, germannormalization, germanstem, greeklowercase, greekstem, englishminimalstem, englishpossessive, kstem, porterstem, spanishlightstem, persiannormalization, finnishlightstem, frenchlightstem, frenchminimalstem, irishlowercase, galicianminimalstem, galicianstem, hindinormalization, hindistem, hungarianlightstem, hunspellstem, indonesianstem, indicnormalization, italianlightstem, latvianstem, asciifolding, capitalization, codepointcount, hyphenatedwords, keepword, keywordmarker, keywordrepeat, length, limittokencount, limittokenposition, removeduplicates, stemmeroverride, trim, truncate, worddelimiter, scandinavianfolding, scandinaviannormalization, edgengram, ngram, norwegianlightstem, norwegianminimalstem, patternreplace, patterncapturegroup, delimitedpayload, numericpayload, tokenoffsetpayload, typeaspayload, portugueselightstem, portugueseminimalstem, portuguesestem, reversestring, russianlightstem, shingle, snowballporter, classic, standard, swedishlightstem, synonym, thaiword, turkishlowercase, elision]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([358F86B7E9419656:BDDBB96D47BDFBAE]:0)
   [junit4]    > 	at org.apache.lucene.analysis.util.AnalysisSPILoader.lookupClass(AnalysisSPILoader.java:122)
   [junit4]    > 	at org.apache.lucene.analysis.util.TokenFilterFactory.lookupClass(TokenFilterFactory.java:42)
   [junit4]    > 	at org.apache.lucene.analysis.core.TestAllAnalyzersHaveFactories.test(TestAllAnalyzersHaveFactories.java:148)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> NOTE: test params are: codec=FastDecompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=56), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=56)), sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=es_SV, timezone=America/Yellowknife
   [junit4]   2> NOTE: Linux 3.8.0-38-generic amd64/Oracle Corporation 1.7.0_60-ea (64-bit)/cpus=8,threads=1,free=66008280,total=126418944
   [junit4]   2> NOTE: All tests run in this JVM: [TestBulgarianStemFilterFactory, TestSnowballPorterFilterFactory, TestTrimFilterFactory, TestHungarianLightStemFilterFactory, TestArmenianAnalyzer, TestEnglishMinimalStemFilterFactory, DateRecognizerSinkTokenizerTest, TokenOffsetPayloadTokenFilterTest, TestKStemmer, QueryAutoStopWordAnalyzerTest, TestSpanishLightStemFilterFactory, EdgeNGramTokenizerTest, DelimitedPayloadTokenFilterTest, TestBulgarianStemmer, TestCompoundWordTokenFilter, TestTurkishAnalyzer, TestTruncateTokenFilterFactory, TestPortugueseAnalyzer, TestPortugueseStemFilter, TestComplexPrefix, TestDanishAnalyzer, TestCJKBigramFilterFactory, TestLatvianStemFilterFactory, TestIndicNormalizer, TestPerFieldAnalyzerWrapper, TestNorwegianMinimalStemFilter, TestHunspellStemFilterFactory, TestCJKWidthFilter, TestCommonGramsFilterFactory, TestCommonGramsQueryFilterFactory, TestCodepointCountFilter, TestHindiFilters, TestClassicAnalyzer, TestFrenchMinimalStemFilterFactory, TestFinnishAnalyzer, NGramTokenFilterTest, TestSegmentingTokenizerBase, TestCollationKeyAnalyzer, TestGalicianStemFilterFactory, TestEnglishMinimalStemFilter, TestHindiAnalyzer, CommonGramsFilterTest, TestMorph, TestGalicianAnalyzer, TestWordnetSynonymParser, TestStandardAnalyzer, TestPortugueseStemFilterFactory, TestArabicFilters, TestStemmerOverrideFilterFactory, TestGreekStemmer, TestFrenchAnalyzer, TestRollingCharBuffer, TestCharArrayMap, TestDictionaryCompoundWordTokenFilterFactory, TestSpanishLightStemFilter, TestIrishLowerCaseFilter, TestCharacterUtils, TestCzechStemmer, TestCapitalizationFilter, TestStemmer, TestUAX29URLEmailTokenizer, TestGreekStemFilterFactory, TestSnowball, TestRandomChains, HTMLStripCharFilterTest, TestAnalyzers, TestSynonymMapFilter, TestPatternReplaceCharFilter, TestRemoveDuplicatesTokenFilter, TestMappingCharFilter, TestAllAnalyzersHaveFactories]
   [junit4] Completed on J1 in 0.21s, 1 test, 1 error <<< FAILURES!

[...truncated 354 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build.xml:539: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:2017: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/analysis/build.xml:106: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/analysis/build.xml:38: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/module-build.xml:60: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1296: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:920: There were test failures: 252 suites, 1342 tests, 1 error, 2 failures, 1 ignored

Total time: 18 minutes 14 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0_60-ea-b14 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.7.0_55) - Build # 10207 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10207/
Java: 64bit/jdk1.7.0_55 -XX:-UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch

Error Message:
shard1 is not consistent.  Got 63 from http://127.0.0.1:49613/collection1lastClient and got 56 from http://127.0.0.1:34073/collection1

Stack Trace:
java.lang.AssertionError: shard1 is not consistent.  Got 63 from http://127.0.0.1:49613/collection1lastClient and got 56 from http://127.0.0.1:34073/collection1
	at __randomizedtesting.SeedInfo.seed([A89C2FE8BFC883DA:297AA1F0C897E3E6]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1130)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1109)
	at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.doTest(ChaosMonkeySafeLeaderTest.java:150)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:865)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10957 lines...]
   [junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
   [junit4]   2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/init-core-data-001
   [junit4]   2> 608909 T2518 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 608909 T2518 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
   [junit4]   2> 608913 T2518 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> 608913 T2518 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 608914 T2519 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 609014 T2518 oasc.ZkTestServer.run start zk server on port:39788
   [junit4]   2> 609015 T2518 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 612706 T2522 oazsp.FileTxnLog.commit WARN fsync-ing the write ahead log in SyncThread:0 took 3689ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
   [junit4]   2> 612707 T2525 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5c516652 name:ZooKeeperConnection Watcher:127.0.0.1:39788 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 612708 T2518 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 612708 T2518 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 612712 T2518 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 612713 T2527 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@182d88db name:ZooKeeperConnection Watcher:127.0.0.1:39788/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 612714 T2518 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 612714 T2518 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 612716 T2518 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 612718 T2518 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 612720 T2518 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 612722 T2518 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 612722 T2518 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 612729 T2518 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 612730 T2518 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 612732 T2518 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 612733 T2518 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 612735 T2518 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 612735 T2518 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 612737 T2518 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 612738 T2518 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 612740 T2518 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 612740 T2518 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 612743 T2518 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 612743 T2518 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 612745 T2518 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 612745 T2518 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 612747 T2518 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 612748 T2518 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 612750 T2518 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 612750 T2518 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 612752 T2518 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 612753 T2518 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 612995 T2518 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 612999 T2518 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:57272
   [junit4]   2> 613000 T2518 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 613000 T2518 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 613000 T2518 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-002
   [junit4]   2> 613001 T2518 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-002/'
   [junit4]   2> 613024 T2518 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-002/solr.xml
   [junit4]   2> 613059 T2518 oasc.CoreContainer.<init> New CoreContainer 1897205401
   [junit4]   2> 613059 T2518 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-002/]
   [junit4]   2> 613060 T2518 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 613060 T2518 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 613061 T2518 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 613061 T2518 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 613061 T2518 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 613061 T2518 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 613061 T2518 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 613062 T2518 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 613062 T2518 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 613062 T2518 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 613063 T2518 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 613063 T2518 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 613063 T2518 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39788/solr
   [junit4]   2> 613063 T2518 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 613064 T2518 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 613066 T2538 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d7198cf name:ZooKeeperConnection Watcher:127.0.0.1:39788 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 613066 T2518 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 613068 T2518 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 613069 T2540 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3a46425f name:ZooKeeperConnection Watcher:127.0.0.1:39788/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 613069 T2518 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 613071 T2518 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 613073 T2518 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 613077 T2518 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 613080 T2518 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 613083 T2518 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 613086 T2518 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 613087 T2518 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57272_
   [junit4]   2> 613088 T2518 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57272_
   [junit4]   2> 613090 T2518 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 613091 T2518 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 613094 T2518 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:57272_
   [junit4]   2> 613094 T2518 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 613096 T2518 oasc.Overseer.start Overseer (id=91681495574511619-127.0.0.1:57272_-n_0000000000) starting
   [junit4]   2> 613098 T2518 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 613105 T2518 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 613106 T2542 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 613107 T2542 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes at 127.0.0.1:57272_
   [junit4]   2> 613109 T2518 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 613111 T2518 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 613116 T2541 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 613116 T2543 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
   [junit4]   2> 613116 T2543 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 613117 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 613118 T2543 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 613118 T2541 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 613119 T2541 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:57272",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:57272_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 613119 T2541 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
   [junit4]   2> 613119 T2541 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 613121 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 613122 T2540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 614118 T2543 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 614118 T2543 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
   [junit4]   2> 614119 T2543 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 614119 T2543 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 614120 T2543 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 614120 T2543 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-002/collection1/'
   [junit4]   2> 614121 T2543 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-002/collection1/lib/.svn/' to classloader
   [junit4]   2> 614121 T2543 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-002/collection1/lib/classes/' to classloader
   [junit4]   2> 614122 T2543 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-002/collection1/lib/README' to classloader
   [junit4]   2> 614179 T2543 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
   [junit4]   2> 614222 T2543 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 614223 T2543 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 614230 T2543 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 614523 T2543 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 614525 T2543 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 614526 T2543 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 614530 T2543 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 614549 T2543 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 614550 T2543 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 614550 T2543 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-002/collection1/, dataDir=./solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-001/control/data/
   [junit4]   2> 614550 T2543 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@784b3598
   [junit4]   2> 614551 T2543 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-001/control/data
   [junit4]   2> 614551 T2543 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-001/control/data/index/
   [junit4]   2> 614552 T2543 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
   [junit4]   2> 614552 T2543 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-001/control/data/index
   [junit4]   2> 614552 T2543 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=15, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=47.3701171875, floorSegmentMB=1.3779296875, forceMergeDeletesPctAllowed=11.295993709071814, segmentsPerTier=12.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7280881276462304
   [junit4]   2> 614557 T2543 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
   [junit4]   2> 614558 T2543 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 614562 T2543 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 614562 T2543 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 614563 T2543 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 614563 T2543 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 614563 T2543 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 614563 T2543 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 614564 T2543 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 614564 T2543 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 614564 T2543 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 614564 T2543 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 614565 T2543 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 614565 T2543 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 614565 T2543 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 614565 T2543 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 614566 T2543 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 614566 T2543 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 614566 T2543 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 614568 T2543 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 614570 T2543 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 614571 T2543 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 614572 T2543 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=727992134, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 614574 T2543 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
   [junit4]   2> 614574 T2543 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 614575 T2543 oass.SolrIndexSearcher.<init> Opening Searcher@c2b6959[collection1] main
   [junit4]   2> 614575 T2543 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 614576 T2543 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 614576 T2543 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 614577 T2543 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 614577 T2543 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 614577 T2543 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 614578 T2543 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 614578 T2543 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 614578 T2543 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
   [junit4]   2> 614579 T2543 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 614582 T2544 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@c2b6959[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 614585 T2543 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 614592 T2547 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57272 collection:control_collection shard:shard1
   [junit4]   2> 614592 T2518 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 614593 T2518 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 614593 T2547 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 614594 T2518 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 614595 T2549 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5fdeb4f name:ZooKeeperConnection Watcher:127.0.0.1:39788/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 614595 T2518 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 614596 T2518 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 614599 T2547 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 614599 T2518 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 614603 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 614604 T2547 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 614604 T2547 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C1875 name=collection1 org.apache.solr.core.SolrCore@63204618 url=http://127.0.0.1:57272/collection1 node=127.0.0.1:57272_ C1875_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:57272, core=collection1, node_name=127.0.0.1:57272_}
   [junit4]   2> 614604 T2547 C1875 P57272 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:57272/collection1/
   [junit4]   2> 614604 T2547 C1875 P57272 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 614605 T2547 C1875 P57272 oasc.SyncStrategy.syncToMe http://127.0.0.1:57272/collection1/ has no replicas
   [junit4]   2> 614605 T2547 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:57272/collection1/ shard1
   [junit4]   2> 614605 T2547 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 614606 T2541 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 614610 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 614612 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 614614 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 614716 T2540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 614716 T2549 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 614762 T2547 oasc.ZkController.register We are http://127.0.0.1:57272/collection1/ and leader is http://127.0.0.1:57272/collection1/
   [junit4]   2> 614763 T2547 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57272
   [junit4]   2> 614763 T2547 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 614763 T2547 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
   [junit4]   2> 614763 T2547 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 614764 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 614765 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 614765 T2547 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 614765 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 614766 T2541 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 614767 T2541 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:57272",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:57272_",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 614770 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 614835 T2518 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
   [junit4]   2> 614835 T2518 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 614837 T2518 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:56277
   [junit4]   2> 614838 T2518 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 614839 T2518 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 614839 T2518 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-003
   [junit4]   2> 614840 T2518 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-003/'
   [junit4]   2> 614853 T2518 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-003/solr.xml
   [junit4]   2> 614876 T2540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 614876 T2549 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 614889 T2518 oasc.CoreContainer.<init> New CoreContainer 1223958474
   [junit4]   2> 614889 T2518 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-003/]
   [junit4]   2> 614890 T2518 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 614891 T2518 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 614891 T2518 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 614891 T2518 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 614891 T2518 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 614892 T2518 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 614892 T2518 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 614892 T2518 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 614892 T2518 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 614893 T2518 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 614893 T2518 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 614893 T2518 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 614894 T2518 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39788/solr
   [junit4]   2> 614894 T2518 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 614895 T2518 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 614896 T2560 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13532c31 name:ZooKeeperConnection Watcher:127.0.0.1:39788 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 614896 T2518 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 614899 T2518 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 614900 T2562 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4985288c name:ZooKeeperConnection Watcher:127.0.0.1:39788/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 614900 T2518 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 614906 T2518 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 615918 T2518 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56277_
   [junit4]   2> 615920 T2518 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:56277_
   [junit4]   2> 615932 T2563 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 615932 T2563 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 615933 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 615933 T2563 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 615933 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 615933 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 615935 T2541 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 615936 T2541 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:56277",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:56277_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 615937 T2541 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3]
   [junit4]   2> 615937 T2541 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
   [junit4]   2> 615939 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 616041 T2540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 616041 T2562 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 616041 T2549 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 616933 T2563 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 616934 T2563 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 616934 T2563 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 616935 T2563 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 616936 T2563 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 616936 T2563 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-003/collection1/'
   [junit4]   2> 616937 T2563 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-003/collection1/lib/.svn/' to classloader
   [junit4]   2> 616937 T2563 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-003/collection1/lib/classes/' to classloader
   [junit4]   2> 616937 T2563 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-003/collection1/lib/README' to classloader
   [junit4]   2> 616974 T2563 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
   [junit4]   2> 617021 T2563 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 617023 T2563 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 617028 T2563 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 617336 T2563 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 617338 T2563 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 617340 T2563 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 617343 T2563 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 617361 T2563 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 617362 T2563 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 617362 T2563 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-003/collection1/, dataDir=./solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-001/jetty1/
   [junit4]   2> 617362 T2563 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@784b3598
   [junit4]   2> 617362 T2563 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-001/jetty1
   [junit4]   2> 617363 T2563 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-001/jetty1/index/
   [junit4]   2> 617363 T2563 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-001/jetty1/index' doesn't exist. Creating new index...
   [junit4]   2> 617364 T2563 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-001/jetty1/index
   [junit4]   2> 617364 T2563 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=15, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=47.3701171875, floorSegmentMB=1.3779296875, forceMergeDeletesPctAllowed=11.295993709071814, segmentsPerTier=12.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7280881276462304
   [junit4]   2> 617368 T2563 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
   [junit4]   2> 617368 T2563 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 617372 T2563 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 617373 T2563 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 617373 T2563 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 617373 T2563 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 617374 T2563 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 617374 T2563 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 617374 T2563 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 617374 T2563 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 617375 T2563 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 617375 T2563 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 617376 T2563 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 617376 T2563 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 617376 T2563 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 617376 T2563 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 617377 T2563 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 617377 T2563 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 617377 T2563 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 617380 T2563 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 617382 T2563 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 617382 T2563 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 617383 T2563 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=727992134, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 617385 T2563 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
   [junit4]   2> 617385 T2563 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 617386 T2563 oass.SolrIndexSearcher.<init> Opening Searcher@32f0a4aa[collection1] main
   [junit4]   2> 617386 T2563 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 617387 T2563 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 617387 T2563 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 617388 T2563 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 617388 T2563 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 617388 T2563 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 617389 T2563 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 617389 T2563 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 617389 T2563 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
   [junit4]   2> 617389 T2563 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 617392 T2564 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@32f0a4aa[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 617395 T2563 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 617395 T2567 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:56277 collection:collection1 shard:shard3
   [junit4]   2> 617396 T2518 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 617396 T2518 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 617396 T2567 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
   [junit4]   2> 617401 T2567 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard3
   [junit4]   2> 617404 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 617404 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 617404 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 617404 T2567 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 617405 T2567 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> 617405 T2541 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> ASYNC  NEW_CORE C1876 name=collection1 org.apache.solr.core.SolrCore@76456b6a url=http://127.0.0.1:56277/collection1 node=127.0.0.1:56277_ C1876_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:56277, core=collection1, node_name=127.0.0.1:56277_}
   [junit4]   2> 617405 T2567 C1876 P56277 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:56277/collection1/
   [junit4]   2> 617406 T2567 C1876 P56277 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 617406 T2567 C1876 P56277 oasc.SyncStrategy.syncToMe http://127.0.0.1:56277/collection1/ has no replicas
   [junit4]   2> 617406 T2567 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:56277/collection1/ shard3
   [junit4]   2> 617406 T2567 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3
   [junit4]   2> 617409 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 617411 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 617414 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 617516 T2540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 617516 T2562 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 617516 T2549 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 617562 T2567 oasc.ZkController.register We are http://127.0.0.1:56277/collection1/ and leader is http://127.0.0.1:56277/collection1/
   [junit4]   2> 617562 T2567 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:56277
   [junit4]   2> 617562 T2567 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 617563 T2567 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
   [junit4]   2> 617563 T2567 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 617564 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 617564 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 617564 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 617565 T2567 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 617565 T2541 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 617566 T2541 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:56277",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:56277_",
   [junit4]   2> 	  "shard":"shard3",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 617568 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 617619 T2518 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
   [junit4]   2> 617620 T2518 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 617621 T2518 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:45005
   [junit4]   2> 617623 T2518 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 617623 T2518 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 617623 T2518 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-004
   [junit4]   2> 617623 T2518 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-004/'
   [junit4]   2> 617634 T2518 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-004/solr.xml
   [junit4]   2> 617674 T2549 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 617674 T2562 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 617675 T2540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 617682 T2518 oasc.CoreContainer.<init> New CoreContainer 1935723354
   [junit4]   2> 617682 T2518 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-004/]
   [junit4]   2> 617683 T2518 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 617684 T2518 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 617684 T2518 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 617684 T2518 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 617684 T2518 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 617684 T2518 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 617685 T2518 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 617685 T2518 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 617685 T2518 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 617686 T2518 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 617686 T2518 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 617686 T2518 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 617686 T2518 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39788/solr
   [junit4]   2> 617687 T2518 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 617688 T2518 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 617689 T2578 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@57c6bce3 name:ZooKeeperConnection Watcher:127.0.0.1:39788 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 617689 T2518 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 617691 T2518 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 617693 T2580 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3eca6705 name:ZooKeeperConnection Watcher:127.0.0.1:39788/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 617693 T2518 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 617701 T2518 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 618704 T2518 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45005_
   [junit4]   2> 618705 T2518 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45005_
   [junit4]   2> 618714 T2581 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 618714 T2581 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 618715 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 618715 T2581 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 618715 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 618716 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 618717 T2541 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 618717 T2541 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:45005",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:45005_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 618717 T2541 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
   [junit4]   2> 618718 T2541 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 618720 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 618821 T2580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 618821 T2549 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 618821 T2540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 618821 T2562 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 619716 T2581 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 619716 T2581 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 619717 T2581 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 619717 T2581 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 619718 T2581 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 619718 T2581 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-004/collection1/'
   [junit4]   2> 619718 T2581 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-004/collection1/lib/.svn/' to classloader
   [junit4]   2> 619719 T2581 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-004/collection1/lib/classes/' to classloader
   [junit4]   2> 619719 T2581 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-004/collection1/lib/README' to classloader
   [junit4]   2> 619765 T2581 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
   [junit4]   2> 619820 T2581 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 619822 T2581 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 619827 T2581 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 620132 T2581 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 620135 T2581 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 620136 T2581 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 620141 T2581 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 620160 T2581 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 620160 T2581 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 620160 T2581 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-004/collection1/, dataDir=./solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-001/jetty2/
   [junit4]   2> 620161 T2581 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@784b3598
   [junit4]   2> 620161 T2581 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-001/jetty2
   [junit4]   2> 620161 T2581 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-001/jetty2/index/
   [junit4]   2> 620162 T2581 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-001/jetty2/index' doesn't exist. Creating new index...
   [junit4]   2> 620162 T2581 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-001/jetty2/index
   [junit4]   2> 620163 T2581 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=15, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=47.3701171875, floorSegmentMB=1.3779296875, forceMergeDeletesPctAllowed=11.295993709071814, segmentsPerTier=12.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7280881276462304
   [junit4]   2> 620166 T2581 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-001/jetty2/index,segFN=segments_1,generation=1}
   [junit4]   2> 620167 T2581 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 620170 T2581 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 620171 T2581 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 620171 T2581 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 620171 T2581 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 620171 T2581 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 620171 T2581 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 620172 T2581 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 620172 T2581 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 620172 T2581 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 620173 T2581 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 620173 T2581 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 620173 T2581 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 620173 T2581 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 620174 T2581 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 620174 T2581 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 620174 T2581 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 620175 T2581 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 620177 T2581 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 620179 T2581 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 620179 T2581 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 620180 T2581 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=727992134, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 620181 T2581 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-001/jetty2/index,segFN=segments_1,generation=1}
   [junit4]   2> 620181 T2581 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 620182 T2581 oass.SolrIndexSearcher.<init> Opening Searcher@1bc1263c[collection1] main
   [junit4]   2> 620182 T2581 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 620183 T2581 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 620183 T2581 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 620184 T2581 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 620184 T2581 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 620184 T2581 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 620185 T2581 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 620185 T2581 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 620185 T2581 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
   [junit4]   2> 620185 T2581 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 620188 T2582 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1bc1263c[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 620191 T2581 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 620191 T2585 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:45005 collection:collection1 shard:shard2
   [junit4]   2> 620192 T2518 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 620192 T2518 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 620192 T2585 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
   [junit4]   2> 620198 T2585 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
   [junit4]   2> 620200 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 620200 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 620200 T2585 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 620200 T2585 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> 620200 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> ASYNC  NEW_CORE C1877 name=collection1 org.apache.solr.core.SolrCore@5de5d7d8 url=http://127.0.0.1:45005/collection1 node=127.0.0.1:45005_ C1877_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:45005, core=collection1, node_name=127.0.0.1:45005_}
   [junit4]   2> 620201 T2585 C1877 P45005 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:45005/collection1/
   [junit4]   2> 620201 T2585 C1877 P45005 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 620201 T2585 C1877 P45005 oasc.SyncStrategy.syncToMe http://127.0.0.1:45005/collection1/ has no replicas
   [junit4]   2> 620202 T2585 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:45005/collection1/ shard2
   [junit4]   2> 620202 T2585 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
   [junit4]   2> 620202 T2541 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 620205 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 620207 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 620209 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 620311 T2549 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 620311 T2562 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 620311 T2580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 620311 T2540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 620357 T2585 oasc.ZkController.register We are http://127.0.0.1:45005/collection1/ and leader is http://127.0.0.1:45005/collection1/
   [junit4]   2> 620358 T2585 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:45005
   [junit4]   2> 620358 T2585 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 620358 T2585 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
   [junit4]   2> 620358 T2585 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 620359 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 620360 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 620360 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 620360 T2585 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 620361 T2541 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 620362 T2541 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:45005",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:45005_",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "core_node_name":"core_node2"}
   [junit4]   2> 620364 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 620416 T2518 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
   [junit4]   2> 620417 T2518 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 620419 T2518 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49613
   [junit4]   2> 620420 T2518 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 620421 T2518 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 620421 T2518 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-005
   [junit4]   2> 620421 T2518 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-005/'
   [junit4]   2> 620435 T2518 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-005/solr.xml
   [junit4]   2> 620469 T2549 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 620469 T2562 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 620469 T2580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 620469 T2540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 620487 T2518 oasc.CoreContainer.<init> New CoreContainer 124593648
   [junit4]   2> 620487 T2518 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-005/]
   [junit4]   2> 620488 T2518 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 620488 T2518 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 620489 T2518 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 620489 T2518 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 620489 T2518 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 620489 T2518 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 620490 T2518 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 620490 T2518 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 620490 T2518 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 620491 T2518 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 620491 T2518 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 620491 T2518 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 620491 T2518 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39788/solr
   [junit4]   2> 620492 T2518 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 620492 T2518 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 620495 T2596 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ed5b35f name:ZooKeeperConnection Watcher:127.0.0.1:39788 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 620495 T2518 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 620497 T2518 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 620498 T2598 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@551d268d name:ZooKeeperConnection Watcher:127.0.0.1:39788/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 620498 T2518 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 620503 T2518 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 621505 T2518 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49613_
   [junit4]   2> 621506 T2518 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49613_
   [junit4]   2> 621515 T2599 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 621515 T2599 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 621516 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 621516 T2599 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 621516 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 621516 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 621517 T2541 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 621518 T2541 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:49613",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:49613_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 621518 T2541 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
   [junit4]   2> 621518 T2541 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 621520 T2540 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 621622 T2562 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 621622 T2540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 621622 T2549 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 621622 T2580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 621622 T2598 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 622516 T2599 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 622517 T2599 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 622518 T2599 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 622518 T2599 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 622519 T2599 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 622519 T2599 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-005/collection1/'
   [junit4]   2> 622520 T2599 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-005/collection1/lib/.svn/' to classloader
   [junit4]   2> 622520 T2599 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-005/collection1/lib/classes/' to classloader
   [junit4]   2> 622521 T2599 oasc.SolrResourceLoader.replaceClassLoader Adding '

[...truncated too long message...]

n: KeeperErrorCode = Session expired for /overseer_elect/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:260)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:257)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> 
   [junit4]   2> ASYNC  NEW_CORE C2142 name=collection1 org.apache.solr.core.SolrCore@c649edd url=http://127.0.0.1:38275/collection1 node=127.0.0.1:38275_ C2142_STATE=coll:collection1 core:collection1 props:{state=active, base_url=http://127.0.0.1:38275, core=collection1, node_name=127.0.0.1:38275_}
   [junit4]   2> 708567 T2762 C2142 P38275 oasc.SyncStrategy.sync WARN Closed, skipping sync up.
   [junit4]   2> 708568 T2762 oasc.ShardLeaderElectionContext.rejoinLeaderElection Not rejoining election because CoreContainer is shutdown
   [junit4]   2> 708568 T2762 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@c649edd
   [junit4]   2> 708570 T2762 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=31,autocommits=0,soft autocommit maxTime=1000ms,soft autocommits=10,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=115,cumulative_deletesById=43,cumulative_deletesByQuery=1,cumulative_errors=0,transaction_logs_total_size=12984,transaction_logs_total_number=1}
   [junit4]   2> 708571 T2762 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 708571 T2762 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 708572 T2762 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 708574 T2762 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 708575 T2762 oasc.CachingDirectoryFactory.close Closing StandardDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 708575 T2762 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-001/jetty11/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-001/jetty11/index;done=false>>]
   [junit4]   2> 708575 T2762 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-001/jetty11/index
   [junit4]   2> 708576 T2762 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-001/jetty11 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-001/jetty11;done=false>>]
   [junit4]   2> 708576 T2762 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001/tempDir-001/jetty11
   [junit4]   2> 708577 T2762 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 708577 T2762 oasc.LeaderElector$ElectionWatcher.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:260)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:257)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> 
   [junit4]   2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.ChaosMonkeySafeLeaderTest-A89C2FE8BFC883DA-001
   [junit4]   2> NOTE: test params are: codec=Lucene45, sim=DefaultSimilarity, locale=ga_IE, timezone=America/Iqaluit
   [junit4]   2> NOTE: Linux 3.8.0-38-generic amd64/Oracle Corporation 1.7.0_55 (64-bit)/cpus=8,threads=1,free=103268888,total=270180352
   [junit4]   2> NOTE: All tests run in this JVM: [TestExtendedDismaxParser, TestNonNRTOpen, TestSolrDeletionPolicy1, ClusterStateUpdateTest, TestTrie, JsonLoaderTest, LoggingHandlerTest, TestBM25SimilarityFactory, CoreAdminHandlerTest, HdfsCollectionsAPIDistributedZkTest, PreAnalyzedUpdateProcessorTest, EnumFieldTest, IndexSchemaTest, SchemaVersionSpecificBehaviorTest, TestInitQParser, BasicFunctionalityTest, SoftAutoCommitTest, TestMaxScoreQueryParser, SolrInfoMBeanTest, AtomicUpdatesTest, BasicDistributedZkTest, TestSearcherReuse, TestSimpleQParserPlugin, AliasIntegrationTest, TestJmxMonitoredMap, TestHighlightDedupGrouping, TestShortCircuitedRequests, SynonymTokenizerTest, TestRestManager, TestAnalyzeInfixSuggestions, TestOverriddenPrefixQueryForCustomFieldType, TestLMJelinekMercerSimilarityFactory, TestExpandComponent, SpatialFilterTest, TestWriterPerf, TestImplicitCoreProperties, ScriptEngineTest, TestPostingsSolrHighlighter, TestJoin, DistribCursorPagingTest, ShowFileRequestHandlerTest, ZkCLITest, TestFunctionQuery, TestCustomSort, HdfsSyncSliceTest, TestUtils, TestElisionMultitermQuery, TestDocBasedVersionConstraints, SolrXmlInZkTest, WordBreakSolrSpellCheckerTest, HdfsBasicDistributedZkTest, URLClassifyProcessorTest, DirectUpdateHandlerOptimizeTest, TestSuggestSpellingConverter, TestNoOpRegenerator, CursorMarkTest, SuggestComponentTest, CursorPagingTest, TestLuceneMatchVersion, ChangedSchemaMergeTest, UpdateParamsTest, EchoParamsTest, TestFieldSortValues, DocExpirationUpdateProcessorFactoryTest, TermsComponentTest, TestDefaultSimilarityFactory, TestSolrXmlPersistence, TestFieldTypeResource, TestRealTimeGet, BinaryUpdateRequestHandlerTest, FastVectorHighlighterTest, SyncSliceTest, DocumentBuilderTest, TestCharFilters, LukeRequestHandlerTest, TestFileDictionaryLookup, SearchHandlerTest, ResponseLogComponentTest, TestMergePolicyConfig, QueryParsingTest, SliceStateTest, TestDynamicFieldResource, PolyFieldTest, TestPHPSerializedResponseWriter, TestFaceting, TestPseudoReturnFields, ChaosMonkeyNothingIsSafeTest, ParsingFieldUpdateProcessorsTest, SpellPossibilityIteratorTest, FieldFacetExtrasTest, ExternalFileFieldSortTest, DistributedQueryComponentCustomSortTest, TestSchemaVersionResource, TestValueSourceCache, ShardRoutingTest, BadComponentTest, ChaosMonkeySafeLeaderTest]
   [junit4] Completed on J0 in 100.55s, 1 test, 1 failure <<< FAILURES!

[...truncated 670 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:496: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1296: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:920: There were test failures: 398 suites, 1669 tests, 1 failure, 37 ignored (16 assumptions)

Total time: 55 minutes 26 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0_55 -XX:-UseCompressedOops -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure