You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2014/11/22 22:02:52 UTC

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 681 - Failure

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/681/

2 tests failed.
REGRESSION:  org.apache.lucene.index.TestIndexWriterOutOfMemory.testCheckpoint

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([DFAC7CC0C8C15945]:0)


FAILED:  junit.framework.TestSuite.org.apache.lucene.index.TestIndexWriterOutOfMemory

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([DFAC7CC0C8C15945]:0)




Build Log:
[...truncated 1859 lines...]
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterOutOfMemory
   [junit4]   2> nov 22, 2014 4:46:06 PM com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
   [junit4]   2> WARNING: Suite execution timed out: org.apache.lucene.index.TestIndexWriterOutOfMemory
   [junit4]   2> ==== jstack at approximately timeout time ====
   [junit4]   2> "TEST-TestIndexWriterOutOfMemory.testCheckpoint-seed#[DFAC7CC0C8C15945]" ID=5621 RUNNABLE
   [junit4]   2> 	at java.lang.Throwable.getStackTraceElement(Native Method)
   [junit4]   2> 	at java.lang.Throwable.getOurStackTrace(Throwable.java:827)
   [junit4]   2> 	- locked java.lang.Exception@8c8fd6
   [junit4]   2> 	at java.lang.Throwable.getStackTrace(Throwable.java:816)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterOutOfMemory$3.eval(TestIndexWriterOutOfMemory.java:257)
   [junit4]   2> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowDeterministicException(MockDirectoryWrapper.java:960)
   [junit4]   2> 	- locked org.apache.lucene.store.MockDirectoryWrapper@c372ae2
   [junit4]   2> 	at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:128)
   [junit4]   2> 	at org.apache.lucene.store.MockIndexOutputWrapper.writeByte(MockIndexOutputWrapper.java:111)
   [junit4]   2> 	at org.apache.lucene.codecs.simpletext.SimpleTextUtil.write(SimpleTextUtil.java:49)
   [junit4]   2> 	at org.apache.lucene.codecs.simpletext.SimpleTextFieldInfosFormat.write(SimpleTextFieldInfosFormat.java:189)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:118)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:439)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:511)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:622)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2731)
   [junit4]   2> 	- locked java.lang.Object@79e061b1
   [junit4]   2> 	- locked java.lang.Object@7a2de9f9
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2886)
   [junit4]   2> 	- locked java.lang.Object@7a2de9f9
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2853)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterOutOfMemory.doTest(TestIndexWriterOutOfMemory.java:177)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterOutOfMemory.testCheckpoint(TestIndexWriterOutOfMemory.java:253)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> "SUITE-TestIndexWriterOutOfMemory-seed#[DFAC7CC0C8C15945]" ID=5620 RUNNABLE
   [junit4]   2> 	at sun.management.ThreadImpl.dumpThreads0(Native Method)
   [junit4]   2> 	at sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:446)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:671)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$900(ThreadLeakControl.java:62)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:412)
   [junit4]   2> 	- locked java.lang.Object@5c5c1919
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:642)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:129)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$1.run(RandomizedRunner.java:559)
   [junit4]   2> 
   [junit4]   2> "JUnit4-serializer-daemon" ID=9 TIMED_WAITING
   [junit4]   2> 	at java.lang.Thread.sleep(Native Method)
   [junit4]   2> 	at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:47)
   [junit4]   2> 
   [junit4]   2> "Signal Dispatcher" ID=4 RUNNABLE
   [junit4]   2> 
   [junit4]   2> "Finalizer" ID=3 WAITING on java.lang.ref.ReferenceQueue$Lock@2273e7ae
   [junit4]   2> 	at java.lang.Object.wait(Native Method)
   [junit4]   2> 	- waiting on java.lang.ref.ReferenceQueue$Lock@2273e7ae
   [junit4]   2> 	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
   [junit4]   2> 	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
   [junit4]   2> 	at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
   [junit4]   2> 
   [junit4]   2> "Reference Handler" ID=2 WAITING on java.lang.ref.Reference$Lock@676f622d
   [junit4]   2> 	at java.lang.Object.wait(Native Method)
   [junit4]   2> 	- waiting on java.lang.ref.Reference$Lock@676f622d
   [junit4]   2> 	at java.lang.Object.wait(Object.java:503)
   [junit4]   2> 	at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
   [junit4]   2> 
   [junit4]   2> "main" ID=1 WAITING on com.carrotsearch.randomizedtesting.RandomizedRunner$1@27b2476d
   [junit4]   2> 	at java.lang.Object.wait(Native Method)
   [junit4]   2> 	- waiting on com.carrotsearch.randomizedtesting.RandomizedRunner$1@27b2476d
   [junit4]   2> 	at java.lang.Thread.join(Thread.java:1281)
   [junit4]   2> 	at java.lang.Thread.join(Thread.java:1355)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:569)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:435)
   [junit4]   2> 	at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:199)
   [junit4]   2> 	at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:310)
   [junit4]   2> 	at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:12)
   [junit4]   2> 
   [junit4]   2> ^^==============================================
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestIndexWriterOutOfMemory -Dtests.method=testCheckpoint -Dtests.seed=DFAC7CC0C8C15945 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=sl_SI -Dtests.timezone=America/Halifax -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   7200s J1 | TestIndexWriterOutOfMemory.testCheckpoint <<<
   [junit4]    > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([DFAC7CC0C8C15945]:0)
   [junit4]   2> nov 22, 2014 4:46:07 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> nov 22, 2014 4:46:27 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.index.TestIndexWriterOutOfMemory: 
   [junit4]   2>    1) Thread[id=5621, name=TEST-TestIndexWriterOutOfMemory.testCheckpoint-seed#[DFAC7CC0C8C15945], state=RUNNABLE, group=TGRP-TestIndexWriterOutOfMemory]
   [junit4]   2>         at java.lang.Throwable.getStackTraceElement(Native Method)
   [junit4]   2>         at java.lang.Throwable.getOurStackTrace(Throwable.java:827)
   [junit4]   2>         at java.lang.Throwable.getStackTrace(Throwable.java:816)
   [junit4]   2>         at org.apache.lucene.index.TestIndexWriterOutOfMemory$3.eval(TestIndexWriterOutOfMemory.java:257)
   [junit4]   2>         at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowDeterministicException(MockDirectoryWrapper.java:960)
   [junit4]   2>         at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:128)
   [junit4]   2>         at org.apache.lucene.store.MockIndexOutputWrapper.writeByte(MockIndexOutputWrapper.java:111)
   [junit4]   2>         at org.apache.lucene.codecs.simpletext.SimpleTextUtil.write(SimpleTextUtil.java:49)
   [junit4]   2>         at org.apache.lucene.codecs.simpletext.SimpleTextStoredFieldsWriter.write(SimpleTextStoredFieldsWriter.java:187)
   [junit4]   2>         at org.apache.lucene.codecs.simpletext.SimpleTextStoredFieldsWriter.writeField(SimpleTextStoredFieldsWriter.java:89)
   [junit4]   2>         at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:374)
   [junit4]   2>         at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:311)
   [junit4]   2>         at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:240)
   [junit4]   2>         at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:455)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1398)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1133)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1118)
   [junit4]   2>         at org.apache.lucene.index.TestIndexWriterOutOfMemory.doTest(TestIndexWriterOutOfMemory.java:130)
   [junit4]   2>         at org.apache.lucene.index.TestIndexWriterOutOfMemory.testCheckpoint(TestIndexWriterOutOfMemory.java:253)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>         at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
   [junit4]   2>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> nov 22, 2014 4:46:27 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=5621, name=TEST-TestIndexWriterOutOfMemory.testCheckpoint-seed#[DFAC7CC0C8C15945], state=RUNNABLE, group=TGRP-TestIndexWriterOutOfMemory]
   [junit4]   2> nov 22, 2014 4:46:30 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> SEVERE: There are still zombie threads that couldn't be terminated:
   [junit4]   2>    1) Thread[id=5621, name=TEST-TestIndexWriterOutOfMemory.testCheckpoint-seed#[DFAC7CC0C8C15945], state=RUNNABLE, group=TGRP-TestIndexWriterOutOfMemory]
   [junit4]   2>         at java.lang.Throwable.getStackTraceElement(Native Method)
   [junit4]   2>         at java.lang.Throwable.getOurStackTrace(Throwable.java:827)
   [junit4]   2>         at java.lang.Throwable.getStackTrace(Throwable.java:816)
   [junit4]   2>         at org.apache.lucene.index.TestIndexWriterOutOfMemory$3.eval(TestIndexWriterOutOfMemory.java:257)
   [junit4]   2>         at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowDeterministicException(MockDirectoryWrapper.java:960)
   [junit4]   2>         at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:128)
   [junit4]   2>         at org.apache.lucene.store.MockIndexOutputWrapper.writeByte(MockIndexOutputWrapper.java:111)
   [junit4]   2>         at org.apache.lucene.codecs.simpletext.SimpleTextUtil.write(SimpleTextUtil.java:49)
   [junit4]   2>         at org.apache.lucene.codecs.simpletext.SimpleTextDocValuesWriter.doAddBinary(SimpleTextDocValuesWriter.java:170)
   [junit4]   2>         at org.apache.lucene.codecs.simpletext.SimpleTextDocValuesWriter.addSortedNumericField(SimpleTextDocValuesWriter.java:278)
   [junit4]   2>         at org.apache.lucene.index.SortedNumericDocValuesWriter.flush(SortedNumericDocValuesWriter.java:111)
   [junit4]   2>         at org.apache.lucene.index.DefaultIndexingChain.writeDocValues(DefaultIndexingChain.java:142)
   [junit4]   2>         at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:93)
   [junit4]   2>         at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:439)
   [junit4]   2>         at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:511)
   [junit4]   2>         at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:622)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2731)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2886)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2853)
   [junit4]   2>         at org.apache.lucene.index.TestIndexWriterOutOfMemory.doTest(TestIndexWriterOutOfMemory.java:177)
   [junit4]   2>         at org.apache.lucene.index.TestIndexWriterOutOfMemory.testCheckpoint(TestIndexWriterOutOfMemory.java:253)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>         at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
   [junit4]   2>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=SimpleText, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {text_payloads=IB LL-L1, text_vectors=DFR I(ne)Z(0.3), text1=DFR I(ne)3(800.0)}, locale=sl_SI, timezone=America/Halifax
   [junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_65 (64-bit)/cpus=16,threads=2,free=23754728,total=88080384
   [junit4]   2> NOTE: All tests run in this JVM: [TestIndexWriterMerging, TestGraphTokenizers, TestSameTokenSamePosition, TestOfflineSorter, TestParallelLeafReader, TestSparseFixedBitSet, TestConjunctions, TestPostingsOffsets, TestAssertingDocValuesFormat, TestSort, TestDeterminism, TestSimilarity, TestPhrasePrefixQuery, TestTimeLimitingCollector, TestDisjunctionMaxQuery, Nested1, TestPackedInts, TestFuzzyQuery, TestParallelCompositeReader, TestCachingWrapperFilter, TestRoaringDocIdSet, TestMultiLevelSkipList, TestAutomatonQueryUnicode, TestPositiveScoresOnlyCollector, TestTopDocsMerge, TestTopFieldCollector, TestAssertingPostingsFormat, TestSpanFirstQuery, TestOperations, TestThreadedForceMerge, TestQueryWrapperFilter, TestLongPostings, TestBasics, TestLevenshteinAutomata, TestDateFilter, TestSumDocFreq, TestLock, TestBooleanUnevenly, TestLucene50CompoundFormat, TestTermdocPerf, TestBlockPostingsFormat, TestBooleanQuery, TestMinShouldMatch2, TestNormsFormat, TestIndexFileDeleter, TestSimpleExplanationsOfNonMatches, TestStressAdvance, TestTryDelete, TestSloppyPhraseQuery, TestCrashCausesCorruptIndex, TestPerFieldDocValuesFormat, TestIndexWriterReader, TestTermVectorsFormat, TestLockFactory, TestPhraseQuery, TestBagOfPositions, TestOmitNorms, TestFixedBitSet, TestComplexExplanations, TestMockAnalyzer, TestFieldCacheRewriteMethod, TestCrash, TestToken, TestFieldsReader, TestDocValuesIndexing, TestPayloadTermQuery, TestBooleanMinShouldMatch, TestTermsEnum2, TestSpans, TestCustomNorms, TestMixedCodecs, TestTransactionRollback, TestFlex, TestUTF32ToUTF8, TestSpanExplanations, TestStressIndexing2, TestLazyProxSkipping, TestDocsAndPositions, TestRamUsageEstimatorOnWildAnimals, TestFieldCacheRangeFilter, TestBagOfPostings, TestCheckIndex, TestPayloadsOnVectors, TestBooleanQueryVisitSubscorers, TestCompiledAutomaton, TestPrefixQuery, Before3, Before3, TestBinaryTerms, TestPositionIncrement, TestIdentityHashSet, TestSimilarityProvider, TestSetOnce, TestCachingTokenFilter, TestPriorityQueue, TestDocCount, TestCachingCollector, NestedSetupChain, NestedTeardownChain, TestFieldCacheTermsFilter, TestTotalHitCountCollector, TestRecyclingByteBlockAllocator, TestRamUsageEstimator, Nested1, TestIsCurrent, TestNoMergePolicy, TestNoMergeScheduler, TestNamedSPILoader, TestIndexWriterLockRelease, TestBytesRef, TestCharFilter, TestTwoPhaseCommitTool, TestNot, TestMockCharFilter, TestIndexWriterOnJRECrash, TestPrefixFilter, Nested1, Test4GBStoredFields, TestCodecUtil, TestDocValues, TestExceedMaxTermLength, TestFieldReuse, TestIndexReaderClose, TestIndexWriterExceptions2, TestIndexWriterOutOfMemory]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestIndexWriterOutOfMemory -Dtests.seed=DFAC7CC0C8C15945 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=sl_SI -Dtests.timezone=America/Halifax -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s J1 | TestIndexWriterOutOfMemory (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([DFAC7CC0C8C15945]:0)
   [junit4] Completed on J1 in 7223.26s, 1 test, 2 errors <<< FAILURES!

[...truncated 13 lines...]
BUILD FAILED
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:532: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:473: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:61: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/extra-targets.xml:39: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:49: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1359: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:966: There were test failures: 406 suites, 3323 tests, 1 suite-level error, 1 error, 40 ignored (30 assumptions)

Total time: 162 minutes 56 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #680
Archived 5 artifacts
Archive block size is 32768
Received 0 blocks and 3652859 bytes
Compression is 0.0%
Took 1.1 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Re: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 681 - Failure

Posted by Michael McCandless <lu...@mikemccandless.com>.
I committed a fix.

It wasn't actually deadlock: the test was plodding along with
SimpleText and too many iterations.  I just cut the iterations ...

Mike McCandless

http://blog.mikemccandless.com


On Sat, Nov 22, 2014 at 4:02 PM, Apache Jenkins Server
<je...@builds.apache.org> wrote:
> Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/681/
>
> 2 tests failed.
> REGRESSION:  org.apache.lucene.index.TestIndexWriterOutOfMemory.testCheckpoint
>
> Error Message:
> Test abandoned because suite timeout was reached.
>
> Stack Trace:
> java.lang.Exception: Test abandoned because suite timeout was reached.
>         at __randomizedtesting.SeedInfo.seed([DFAC7CC0C8C15945]:0)
>
>
> FAILED:  junit.framework.TestSuite.org.apache.lucene.index.TestIndexWriterOutOfMemory
>
> Error Message:
> Suite timeout exceeded (>= 7200000 msec).
>
> Stack Trace:
> java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
>         at __randomizedtesting.SeedInfo.seed([DFAC7CC0C8C15945]:0)
>
>
>
>
> Build Log:
> [...truncated 1859 lines...]
>    [junit4] Suite: org.apache.lucene.index.TestIndexWriterOutOfMemory
>    [junit4]   2> nov 22, 2014 4:46:06 PM com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
>    [junit4]   2> WARNING: Suite execution timed out: org.apache.lucene.index.TestIndexWriterOutOfMemory
>    [junit4]   2> ==== jstack at approximately timeout time ====
>    [junit4]   2> "TEST-TestIndexWriterOutOfMemory.testCheckpoint-seed#[DFAC7CC0C8C15945]" ID=5621 RUNNABLE
>    [junit4]   2>        at java.lang.Throwable.getStackTraceElement(Native Method)
>    [junit4]   2>        at java.lang.Throwable.getOurStackTrace(Throwable.java:827)
>    [junit4]   2>        - locked java.lang.Exception@8c8fd6
>    [junit4]   2>        at java.lang.Throwable.getStackTrace(Throwable.java:816)
>    [junit4]   2>        at org.apache.lucene.index.TestIndexWriterOutOfMemory$3.eval(TestIndexWriterOutOfMemory.java:257)
>    [junit4]   2>        at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowDeterministicException(MockDirectoryWrapper.java:960)
>    [junit4]   2>        - locked org.apache.lucene.store.MockDirectoryWrapper@c372ae2
>    [junit4]   2>        at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:128)
>    [junit4]   2>        at org.apache.lucene.store.MockIndexOutputWrapper.writeByte(MockIndexOutputWrapper.java:111)
>    [junit4]   2>        at org.apache.lucene.codecs.simpletext.SimpleTextUtil.write(SimpleTextUtil.java:49)
>    [junit4]   2>        at org.apache.lucene.codecs.simpletext.SimpleTextFieldInfosFormat.write(SimpleTextFieldInfosFormat.java:189)
>    [junit4]   2>        at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:118)
>    [junit4]   2>        at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:439)
>    [junit4]   2>        at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:511)
>    [junit4]   2>        at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:622)
>    [junit4]   2>        at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2731)
>    [junit4]   2>        - locked java.lang.Object@79e061b1
>    [junit4]   2>        - locked java.lang.Object@7a2de9f9
>    [junit4]   2>        at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2886)
>    [junit4]   2>        - locked java.lang.Object@7a2de9f9
>    [junit4]   2>        at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2853)
>    [junit4]   2>        at org.apache.lucene.index.TestIndexWriterOutOfMemory.doTest(TestIndexWriterOutOfMemory.java:177)
>    [junit4]   2>        at org.apache.lucene.index.TestIndexWriterOutOfMemory.testCheckpoint(TestIndexWriterOutOfMemory.java:253)
>    [junit4]   2>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    [junit4]   2>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>    [junit4]   2>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>    [junit4]   2>        at java.lang.reflect.Method.invoke(Method.java:606)
>    [junit4]   2>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
>    [junit4]   2>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
>    [junit4]   2>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
>    [junit4]   2>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
>    [junit4]   2>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>    [junit4]   2>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   2>        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>    [junit4]   2>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>    [junit4]   2>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   2>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   2>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   2>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   2>        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>    [junit4]   2>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>    [junit4]   2>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
>    [junit4]   2>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
>    [junit4]   2>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
>    [junit4]   2>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
>    [junit4]   2>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   2>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>    [junit4]   2>        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>    [junit4]   2>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   2>        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   2>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   2>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   2>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   2>        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>    [junit4]   2>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   2>        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   2>        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>    [junit4]   2>        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   2>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
>    [junit4]   2>
>    [junit4]   2> "SUITE-TestIndexWriterOutOfMemory-seed#[DFAC7CC0C8C15945]" ID=5620 RUNNABLE
>    [junit4]   2>        at sun.management.ThreadImpl.dumpThreads0(Native Method)
>    [junit4]   2>        at sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:446)
>    [junit4]   2>        at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:671)
>    [junit4]   2>        at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$900(ThreadLeakControl.java:62)
>    [junit4]   2>        at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:412)
>    [junit4]   2>        - locked java.lang.Object@5c5c1919
>    [junit4]   2>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:642)
>    [junit4]   2>        at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:129)
>    [junit4]   2>        at com.carrotsearch.randomizedtesting.RandomizedRunner$1.run(RandomizedRunner.java:559)
>    [junit4]   2>
>    [junit4]   2> "JUnit4-serializer-daemon" ID=9 TIMED_WAITING
>    [junit4]   2>        at java.lang.Thread.sleep(Native Method)
>    [junit4]   2>        at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:47)
>    [junit4]   2>
>    [junit4]   2> "Signal Dispatcher" ID=4 RUNNABLE
>    [junit4]   2>
>    [junit4]   2> "Finalizer" ID=3 WAITING on java.lang.ref.ReferenceQueue$Lock@2273e7ae
>    [junit4]   2>        at java.lang.Object.wait(Native Method)
>    [junit4]   2>        - waiting on java.lang.ref.ReferenceQueue$Lock@2273e7ae
>    [junit4]   2>        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
>    [junit4]   2>        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
>    [junit4]   2>        at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
>    [junit4]   2>
>    [junit4]   2> "Reference Handler" ID=2 WAITING on java.lang.ref.Reference$Lock@676f622d
>    [junit4]   2>        at java.lang.Object.wait(Native Method)
>    [junit4]   2>        - waiting on java.lang.ref.Reference$Lock@676f622d
>    [junit4]   2>        at java.lang.Object.wait(Object.java:503)
>    [junit4]   2>        at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
>    [junit4]   2>
>    [junit4]   2> "main" ID=1 WAITING on com.carrotsearch.randomizedtesting.RandomizedRunner$1@27b2476d
>    [junit4]   2>        at java.lang.Object.wait(Native Method)
>    [junit4]   2>        - waiting on com.carrotsearch.randomizedtesting.RandomizedRunner$1@27b2476d
>    [junit4]   2>        at java.lang.Thread.join(Thread.java:1281)
>    [junit4]   2>        at java.lang.Thread.join(Thread.java:1355)
>    [junit4]   2>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:569)
>    [junit4]   2>        at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:435)
>    [junit4]   2>        at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:199)
>    [junit4]   2>        at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:310)
>    [junit4]   2>        at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:12)
>    [junit4]   2>
>    [junit4]   2> ^^==============================================
>    [junit4]   2>
>    [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
>    [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestIndexWriterOutOfMemory -Dtests.method=testCheckpoint -Dtests.seed=DFAC7CC0C8C15945 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=sl_SI -Dtests.timezone=America/Halifax -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
>    [junit4] ERROR   7200s J1 | TestIndexWriterOutOfMemory.testCheckpoint <<<
>    [junit4]    > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
>    [junit4]    >        at __randomizedtesting.SeedInfo.seed([DFAC7CC0C8C15945]:0)
>    [junit4]   2> nov 22, 2014 4:46:07 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
>    [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
>    [junit4]   2> nov 22, 2014 4:46:27 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
>    [junit4]   2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.index.TestIndexWriterOutOfMemory:
>    [junit4]   2>    1) Thread[id=5621, name=TEST-TestIndexWriterOutOfMemory.testCheckpoint-seed#[DFAC7CC0C8C15945], state=RUNNABLE, group=TGRP-TestIndexWriterOutOfMemory]
>    [junit4]   2>         at java.lang.Throwable.getStackTraceElement(Native Method)
>    [junit4]   2>         at java.lang.Throwable.getOurStackTrace(Throwable.java:827)
>    [junit4]   2>         at java.lang.Throwable.getStackTrace(Throwable.java:816)
>    [junit4]   2>         at org.apache.lucene.index.TestIndexWriterOutOfMemory$3.eval(TestIndexWriterOutOfMemory.java:257)
>    [junit4]   2>         at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowDeterministicException(MockDirectoryWrapper.java:960)
>    [junit4]   2>         at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:128)
>    [junit4]   2>         at org.apache.lucene.store.MockIndexOutputWrapper.writeByte(MockIndexOutputWrapper.java:111)
>    [junit4]   2>         at org.apache.lucene.codecs.simpletext.SimpleTextUtil.write(SimpleTextUtil.java:49)
>    [junit4]   2>         at org.apache.lucene.codecs.simpletext.SimpleTextStoredFieldsWriter.write(SimpleTextStoredFieldsWriter.java:187)
>    [junit4]   2>         at org.apache.lucene.codecs.simpletext.SimpleTextStoredFieldsWriter.writeField(SimpleTextStoredFieldsWriter.java:89)
>    [junit4]   2>         at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:374)
>    [junit4]   2>         at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:311)
>    [junit4]   2>         at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:240)
>    [junit4]   2>         at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:455)
>    [junit4]   2>         at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1398)
>    [junit4]   2>         at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1133)
>    [junit4]   2>         at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1118)
>    [junit4]   2>         at org.apache.lucene.index.TestIndexWriterOutOfMemory.doTest(TestIndexWriterOutOfMemory.java:130)
>    [junit4]   2>         at org.apache.lucene.index.TestIndexWriterOutOfMemory.testCheckpoint(TestIndexWriterOutOfMemory.java:253)
>    [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>    [junit4]   2>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>    [junit4]   2>         at java.lang.reflect.Method.invoke(Method.java:606)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
>    [junit4]   2>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>    [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>    [junit4]   2>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>    [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
>    [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   2>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   2>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>    [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
>    [junit4]   2> nov 22, 2014 4:46:27 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
>    [junit4]   2> INFO: Starting to interrupt leaked threads:
>    [junit4]   2>    1) Thread[id=5621, name=TEST-TestIndexWriterOutOfMemory.testCheckpoint-seed#[DFAC7CC0C8C15945], state=RUNNABLE, group=TGRP-TestIndexWriterOutOfMemory]
>    [junit4]   2> nov 22, 2014 4:46:30 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
>    [junit4]   2> SEVERE: There are still zombie threads that couldn't be terminated:
>    [junit4]   2>    1) Thread[id=5621, name=TEST-TestIndexWriterOutOfMemory.testCheckpoint-seed#[DFAC7CC0C8C15945], state=RUNNABLE, group=TGRP-TestIndexWriterOutOfMemory]
>    [junit4]   2>         at java.lang.Throwable.getStackTraceElement(Native Method)
>    [junit4]   2>         at java.lang.Throwable.getOurStackTrace(Throwable.java:827)
>    [junit4]   2>         at java.lang.Throwable.getStackTrace(Throwable.java:816)
>    [junit4]   2>         at org.apache.lucene.index.TestIndexWriterOutOfMemory$3.eval(TestIndexWriterOutOfMemory.java:257)
>    [junit4]   2>         at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowDeterministicException(MockDirectoryWrapper.java:960)
>    [junit4]   2>         at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:128)
>    [junit4]   2>         at org.apache.lucene.store.MockIndexOutputWrapper.writeByte(MockIndexOutputWrapper.java:111)
>    [junit4]   2>         at org.apache.lucene.codecs.simpletext.SimpleTextUtil.write(SimpleTextUtil.java:49)
>    [junit4]   2>         at org.apache.lucene.codecs.simpletext.SimpleTextDocValuesWriter.doAddBinary(SimpleTextDocValuesWriter.java:170)
>    [junit4]   2>         at org.apache.lucene.codecs.simpletext.SimpleTextDocValuesWriter.addSortedNumericField(SimpleTextDocValuesWriter.java:278)
>    [junit4]   2>         at org.apache.lucene.index.SortedNumericDocValuesWriter.flush(SortedNumericDocValuesWriter.java:111)
>    [junit4]   2>         at org.apache.lucene.index.DefaultIndexingChain.writeDocValues(DefaultIndexingChain.java:142)
>    [junit4]   2>         at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:93)
>    [junit4]   2>         at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:439)
>    [junit4]   2>         at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:511)
>    [junit4]   2>         at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:622)
>    [junit4]   2>         at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2731)
>    [junit4]   2>         at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2886)
>    [junit4]   2>         at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2853)
>    [junit4]   2>         at org.apache.lucene.index.TestIndexWriterOutOfMemory.doTest(TestIndexWriterOutOfMemory.java:177)
>    [junit4]   2>         at org.apache.lucene.index.TestIndexWriterOutOfMemory.testCheckpoint(TestIndexWriterOutOfMemory.java:253)
>    [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>    [junit4]   2>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>    [junit4]   2>         at java.lang.reflect.Method.invoke(Method.java:606)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
>    [junit4]   2>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>    [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>    [junit4]   2>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>    [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
>    [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>    [junit4]   2>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   2>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>    [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>    [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>    [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>    [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>    [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
>    [junit4]   2> NOTE: test params are: codec=SimpleText, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {text_payloads=IB LL-L1, text_vectors=DFR I(ne)Z(0.3), text1=DFR I(ne)3(800.0)}, locale=sl_SI, timezone=America/Halifax
>    [junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_65 (64-bit)/cpus=16,threads=2,free=23754728,total=88080384
>    [junit4]   2> NOTE: All tests run in this JVM: [TestIndexWriterMerging, TestGraphTokenizers, TestSameTokenSamePosition, TestOfflineSorter, TestParallelLeafReader, TestSparseFixedBitSet, TestConjunctions, TestPostingsOffsets, TestAssertingDocValuesFormat, TestSort, TestDeterminism, TestSimilarity, TestPhrasePrefixQuery, TestTimeLimitingCollector, TestDisjunctionMaxQuery, Nested1, TestPackedInts, TestFuzzyQuery, TestParallelCompositeReader, TestCachingWrapperFilter, TestRoaringDocIdSet, TestMultiLevelSkipList, TestAutomatonQueryUnicode, TestPositiveScoresOnlyCollector, TestTopDocsMerge, TestTopFieldCollector, TestAssertingPostingsFormat, TestSpanFirstQuery, TestOperations, TestThreadedForceMerge, TestQueryWrapperFilter, TestLongPostings, TestBasics, TestLevenshteinAutomata, TestDateFilter, TestSumDocFreq, TestLock, TestBooleanUnevenly, TestLucene50CompoundFormat, TestTermdocPerf, TestBlockPostingsFormat, TestBooleanQuery, TestMinShouldMatch2, TestNormsFormat, TestIndexFileDeleter, TestSimpleExplanationsOfNonMatches, TestStressAdvance, TestTryDelete, TestSloppyPhraseQuery, TestCrashCausesCorruptIndex, TestPerFieldDocValuesFormat, TestIndexWriterReader, TestTermVectorsFormat, TestLockFactory, TestPhraseQuery, TestBagOfPositions, TestOmitNorms, TestFixedBitSet, TestComplexExplanations, TestMockAnalyzer, TestFieldCacheRewriteMethod, TestCrash, TestToken, TestFieldsReader, TestDocValuesIndexing, TestPayloadTermQuery, TestBooleanMinShouldMatch, TestTermsEnum2, TestSpans, TestCustomNorms, TestMixedCodecs, TestTransactionRollback, TestFlex, TestUTF32ToUTF8, TestSpanExplanations, TestStressIndexing2, TestLazyProxSkipping, TestDocsAndPositions, TestRamUsageEstimatorOnWildAnimals, TestFieldCacheRangeFilter, TestBagOfPostings, TestCheckIndex, TestPayloadsOnVectors, TestBooleanQueryVisitSubscorers, TestCompiledAutomaton, TestPrefixQuery, Before3, Before3, TestBinaryTerms, TestPositionIncrement, TestIdentityHashSet, TestSimilarityProvider, TestSetOnce, TestCachingTokenFilter, TestPriorityQueue, TestDocCount, TestCachingCollector, NestedSetupChain, NestedTeardownChain, TestFieldCacheTermsFilter, TestTotalHitCountCollector, TestRecyclingByteBlockAllocator, TestRamUsageEstimator, Nested1, TestIsCurrent, TestNoMergePolicy, TestNoMergeScheduler, TestNamedSPILoader, TestIndexWriterLockRelease, TestBytesRef, TestCharFilter, TestTwoPhaseCommitTool, TestNot, TestMockCharFilter, TestIndexWriterOnJRECrash, TestPrefixFilter, Nested1, Test4GBStoredFields, TestCodecUtil, TestDocValues, TestExceedMaxTermLength, TestFieldReuse, TestIndexReaderClose, TestIndexWriterExceptions2, TestIndexWriterOutOfMemory]
>    [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
>    [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestIndexWriterOutOfMemory -Dtests.seed=DFAC7CC0C8C15945 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=sl_SI -Dtests.timezone=America/Halifax -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
>    [junit4] ERROR   0.00s J1 | TestIndexWriterOutOfMemory (suite) <<<
>    [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
>    [junit4]    >        at __randomizedtesting.SeedInfo.seed([DFAC7CC0C8C15945]:0)
>    [junit4] Completed on J1 in 7223.26s, 1 test, 2 errors <<< FAILURES!
>
> [...truncated 13 lines...]
> BUILD FAILED
> /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:532: The following error occurred while executing this line:
> /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:473: The following error occurred while executing this line:
> /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:61: The following error occurred while executing this line:
> /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/extra-targets.xml:39: The following error occurred while executing this line:
> /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:49: The following error occurred while executing this line:
> /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1359: The following error occurred while executing this line:
> /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:966: There were test failures: 406 suites, 3323 tests, 1 suite-level error, 1 error, 40 ignored (30 assumptions)
>
> Total time: 162 minutes 56 seconds
> Build step 'Invoke Ant' marked build as failure
> Archiving artifacts
> Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #680
> Archived 5 artifacts
> Archive block size is 32768
> Received 0 blocks and 3652859 bytes
> Compression is 0.0%
> Took 1.1 sec
> Recording test results
> Email was triggered for: Failure
> Sending email for trigger: Failure
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 688 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/688/

10 tests failed.
REGRESSION:  org.apache.solr.spelling.SpellCheckCollatorTest.testEstimatedHitCounts

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([1D09958961FE53F3:2CB22BBCC4C14323]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:729)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:696)
	at org.apache.solr.spelling.SpellCheckCollatorTest.testEstimatedHitCounts(SpellCheckCollatorTest.java:530)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//lst[@name='spellcheck']/lst[@name='collations']/lst[@name='collation']/int[@name='hits' and 6 <= . and . <= 10]
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">6</int></lst><result name="response" numFound="0" start="0"></result><lst name="spellcheck"><lst name="suggestions"><lst name="everother"><int name="numFound">1</int><int name="startOffset">9</int><int name="endOffset">18</int><arr name="suggestion"><str>everyother</str></arr></lst></lst><lst name="collations"><lst name="collation"><str name="collationQuery">teststop:everyother</str><int name="hits">14</int><lst name="misspellingsAndCorrections"><str name="everother">everyother</str></lst></lst></lst></lst>
</response>

	request was:spellcheck=true&spellcheck.dictionary=direct&spellcheck.count=1&spellcheck.collate=true&spellcheck.maxCollationTries=1&spellcheck.maxCollations=1&spellcheck.collateExtendedResults=true&qt=spellCheckCompRH&q=teststop%3Aeverother&spellcheck.collateMaxCollectDocs=5
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:722)
	... 42 more


FAILED:  org.apache.solr.highlight.HighlighterTest.testDisMaxHighlight

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([1D09958961FE53F3:7A1BA0D2020A262]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:729)
	at org.apache.solr.highlight.HighlighterTest.testDisMaxHighlight(HighlighterTest.java:262)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//lst[@name='1']/arr[@name='tv_text']/str
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">64</int></lst><result name="response" numFound="1" start="0"><doc><arr name="tv_text"><str>a long day's night</str></arr><int name="id">1</int><arr name="range_facet_l"><long>1</long></arr><long name="_version_">1486138107267907584</long><int name="intDefault">42</int><arr name="multiDefault"><str>muLti-Default</str></arr><date name="timestamp">2014-11-29T20:09:03.534Z</date></doc></result><lst name="highlighting"><lst name="1"/></lst>
</response>

	request was:rows=200&q=long&q.alt=*:*&qt=dismax&qf=tv_text&start=0&hl.fl=tv_text&hl=true
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:722)
	... 41 more


FAILED:  org.apache.solr.highlight.HighlighterTest.testSubwordWildcardHighlightWithTermOffsets

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([1D09958961FE53F3:22AF714A945B318E]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:729)
	at org.apache.solr.highlight.HighlighterTest.testSubwordWildcardHighlightWithTermOffsets(HighlighterTest.java:834)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//lst[@name='highlighting']/lst[@name='1']/arr[@name='subword_offsets']/str='lorem <em>PowerShot.com</em> ipsum'
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">3</int></lst><result name="response" numFound="1" start="0"><doc><arr name="subword_offsets"><str>lorem PowerShot.com ipsum</str></arr><int name="id">1</int><arr name="range_facet_l"><long>1</long></arr><long name="_version_">1486138107530051584</long><int name="intDefault">42</int><arr name="multiDefault"><str>muLti-Default</str></arr><date name="timestamp">2014-11-29T20:09:03.784Z</date></doc></result><lst name="highlighting"><lst name="1"/></lst>
</response>

	request was:q=subword_offsets:pow*&wt=xml&hl.fl=subword_offsets&hl=true
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:722)
	... 41 more


FAILED:  org.apache.solr.highlight.HighlighterTest.testTermVecMultiValuedHighlight

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([1D09958961FE53F3:B75791E5D41447EB]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:729)
	at org.apache.solr.highlight.HighlighterTest.testTermVecMultiValuedHighlight(HighlighterTest.java:211)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//lst[@name='1']/arr[@name='tv_mv_text']/str[.='a <em>long</em> days night this should be a piece of text which']
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">2</int></lst><result name="response" numFound="1" start="0"><doc><arr name="tv_mv_text"><str>a long days night this should be a piece of text which is is is is is is is is is is is is is is is is is is is is is is is is isis is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is sufficiently lengthly to produce multiple fragments which are not concatenated at all--we want two disjoint long fragments.</str><str>a long days night this should be a piece of text which is is is is is is is is is is is is is is is is is is is is is is is is isis is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is sufficiently lengthly to produce multiple fragments which are not concatenated at all--we want two disjoint long fragments.</str></arr><int name="id">1</int><arr name="range_facet_l"><long>1</long></arr><long name="_version_">1486138107842527232</long><int name="intDefault">42</int><arr name="multiDefault"><str>muLti-Default</str></arr><date name="timestamp">2014-11-29T20:09:04.082Z</date></doc></result><lst name="highlighting"><lst name="1"/></lst>
</response>

	request was:rows=200&q=tv_mv_text:long&qt=standard&start=0&hl.fl=tv_mv_text&hl.snippets=2&hl=true
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:722)
	... 41 more


FAILED:  org.apache.solr.highlight.HighlighterTest.testLongFragment

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([1D09958961FE53F3:85CEE92E6ED2583B]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:729)
	at org.apache.solr.highlight.HighlighterTest.testLongFragment(HighlighterTest.java:496)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//lst[@name='1']/arr[@name='tv_text']/str
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">2</int></lst><result name="response" numFound="1" start="0"><doc><arr name="tv_text"><str>junit: [mkdir] Created dir: /home/klaas/worio/backend/trunk/build-src/solr-nightly/build/test-results [junit] Running org.apache.solr.BasicFunctionalityTest [junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed: 5.36 sec [junit] Running org.apache.solr.ConvertedLegacyTest [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 8.268 sec [junit] Running org.apache.solr.DisMaxRequestHandlerTest [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 1.56 sec [junit] Running org.apache.solr.HighlighterTest [junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed: 4.979 sec [junit] Running org.apache.solr.OutputWriterTest [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.797 sec [junit] Running org.apache.solr.SampleTest [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 1.021 sec [junit] Running org.apache.solr.analysis.TestBufferedTokenStream [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.05 sec [junit] Running org.apache.solr.analysis.TestRemoveDuplicatesTokenFilter [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.054 sec [junit] Running org.apache.solr.analysis.TestSynonymFilter [junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 0.081 sec [junit] Running org.apache.solr.analysis.TestWordDelimiterFilter [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 1.714 sec [junit] Running org.apache.solr.search.TestDocSet [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.788 sec [junit] Running org.apache.solr.util.SolrPluginUtilsTest [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 3.519 sec [junit] Running org.apache.solr.util.TestOpenBitSet [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.533 sec</str></arr><int name="id">1</int><arr name="range_facet_l"><long>1</long></arr><long name="_version_">1486138108039659520</long><int name="intDefault">42</int><arr name="multiDefault"><str>muLti-Default</str></arr><date name="timestamp">2014-11-29T20:09:04.269Z</date></doc></result><lst name="highlighting"><lst name="1"/></lst>
</response>

	request was:rows=200&q=tv_text:dir&qt=standard&start=0&hl.fl=tv_text&hl=true
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:722)
	... 41 more


FAILED:  org.apache.solr.highlight.HighlighterTest.testTermVecMultiValuedHighlight2

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([1D09958961FE53F3:580A855FF2DC6C1]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:729)
	at org.apache.solr.highlight.HighlighterTest.testTermVecMultiValuedHighlight2(HighlighterTest.java:239)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//lst[@name='1']/arr[@name='tv_mv_text']/str[.='a <em>long</em> days night this should be a piece of text which']
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">1</int></lst><result name="response" numFound="1" start="0"><doc><arr name="tv_mv_text"><str>short</str><str>a long days night this should be a piece of text which is is is is is is is is is is is is is is is is is is is is is is is is isis is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is sufficiently lengthly to produce multiple fragments which are not concatenated at all--we want two disjoint long fragments.</str></arr><int name="id">1</int><arr name="range_facet_l"><long>1</long></arr><long name="_version_">1486138108166537216</long><int name="intDefault">42</int><arr name="multiDefault"><str>muLti-Default</str></arr><date name="timestamp">2014-11-29T20:09:04.39Z</date></doc></result><lst name="highlighting"><lst name="1"/></lst>
</response>

	request was:rows=200&q=tv_mv_text:long&qt=standard&start=0&hl.fl=tv_mv_text&hl.snippets=2&hl=true
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:722)
	... 41 more


FAILED:  org.apache.solr.highlight.HighlighterTest.testVariableFragsize

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([1D09958961FE53F3:1A4595F88340FF61]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:729)
	at org.apache.solr.highlight.HighlighterTest.testVariableFragsize(HighlighterTest.java:592)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//lst[@name='1']/arr[@name='tv_text']/str[.='a <em>long</em> days night this should be a piece of text which']
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">2</int></lst><result name="response" numFound="1" start="0"><doc><arr name="tv_text"><str>a long days night this should be a piece of text which is is is is is is is is is is is is is is is is is is is is is is is is isis is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is sufficiently lengthly to produce multiple fragments which are not concatenated at all</str></arr><int name="id">1</int><arr name="range_facet_l"><long>1</long></arr><long name="_version_">1486138108667756544</long><int name="intDefault">42</int><arr name="multiDefault"><str>muLti-Default</str></arr><date name="timestamp">2014-11-29T20:09:04.869Z</date></doc></result><lst name="highlighting"><lst name="1"/></lst>
</response>

	request was:rows=200&q=tv_text:long&qt=standard&start=0&hl.fl=tv_text&hl=true
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:722)
	... 41 more


FAILED:  org.apache.solr.highlight.HighlighterTest.testSubwordWildcardHighlightWithTermOffsets2

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([1D09958961FE53F3:2FAACE00A152A99D]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:729)
	at org.apache.solr.highlight.HighlighterTest.testSubwordWildcardHighlightWithTermOffsets2(HighlighterTest.java:843)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//lst[@name='highlighting']/lst[@name='1']/arr[@name='subword_offsets']/str='lorem <em>PowerShot</em> ipsum'
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">2</int></lst><result name="response" numFound="1" start="0"><doc><arr name="subword_offsets"><str>lorem PowerShot ipsum</str></arr><int name="id">1</int><arr name="range_facet_l"><long>1</long></arr><long name="_version_">1486138109107109888</long><int name="intDefault">42</int><arr name="multiDefault"><str>muLti-Default</str></arr><date name="timestamp">2014-11-29T20:09:05.288Z</date></doc></result><lst name="highlighting"><lst name="1"/></lst>
</response>

	request was:q=subword_offsets:pow*&wt=xml&hl.fl=subword_offsets&hl=true
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:722)
	... 41 more


FAILED:  org.apache.solr.highlight.HighlighterTest.testTermVecHighlight

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([1D09958961FE53F3:3C29ED5868AD87E0]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:729)
	at org.apache.solr.highlight.HighlighterTest.testTermVecHighlight(HighlighterTest.java:145)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//lst[@name='1']/arr[@name='tv_text']/str[.='a <em>long</em> days night this should be a piece of text which']
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">1</int></lst><result name="response" numFound="1" start="0"><doc><arr name="tv_text"><str>a long days night this should be a piece of text which is is is is is is is is is is is is is is is is is is is is is is is is isis is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is sufficiently lengthly to produce multiple fragments which are not concatenated at all--we want two disjoint long fragments.</str></arr><int name="id">1</int><arr name="range_facet_l"><long>1</long></arr><long name="_version_">1486138109594697728</long><int name="intDefault">42</int><arr name="multiDefault"><str>muLti-Default</str></arr><date name="timestamp">2014-11-29T20:09:05.753Z</date></doc></result><lst name="highlighting"><lst name="1"/></lst>
</response>

	request was:rows=200&q=tv_text:long&qt=standard&start=0&hl.fl=tv_text&hl.snippets=2&hl=true
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:722)
	... 41 more


FAILED:  org.apache.solr.highlight.HighlighterTest.testTwoFieldHighlight

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([1D09958961FE53F3:647B508B86880549]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:729)
	at org.apache.solr.highlight.HighlighterTest.testTwoFieldHighlight(HighlighterTest.java:391)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//lst[@name='1']/arr[@name='tv_text']/str
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">2</int></lst><result name="response" numFound="1" start="0"><doc><arr name="t_text"><str>a long day's night</str></arr><int name="id">1</int><arr name="range_facet_l"><long>1</long></arr><arr name="tv_text"><str>a long night's day</str></arr><long name="_version_">1486138109715283968</long><int name="intDefault">42</int><arr name="multiDefault"><str>muLti-Default</str></arr><date name="timestamp">2014-11-29T20:09:05.867Z</date></doc></result><lst name="highlighting"><lst name="1"><arr name="t_text"><str>a &lt;em&gt;long&lt;/em&gt; day's night</str></arr></lst></lst>
</response>

	request was:rows=200&q=t_text:long&qt=standard&start=0&hl.fl=t_text+tv_text&hl=true
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:722)
	... 41 more




Build Log:
[...truncated 10133 lines...]
   [junit4] Suite: org.apache.solr.spelling.SpellCheckCollatorTest
   [junit4]   2> Creating dataDir: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.spelling.SpellCheckCollatorTest-1D09958961FE53F3-001/init-core-data-001
   [junit4]   2> 1506229 T2949 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (true)
   [junit4]   2> 1506230 T2949 oas.SolrTestCaseJ4.initCore ####initCore
   [junit4]   2> 1506231 T2949 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 1506232 T2949 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 1506232 T2949 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 1506232 T2949 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 1506336 T2949 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 1506376 T2949 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-spellcheckcomponent.xml
   [junit4]   2> 1506377 T2949 oass.IndexSchema.readSchema Reading Solr Schema from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml
   [junit4]   2> 1506389 T2949 oass.IndexSchema.readSchema [null] Schema name=test
   [junit4]   2> 1506697 T2949 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1506710 T2949 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1506713 T2949 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1506724 T2949 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1506728 T2949 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1506732 T2949 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1506733 T2949 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1506733 T2949 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1506734 T2949 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1506734 T2949 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1506735 T2949 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1506735 T2949 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1506735 T2949 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr
   [junit4]   2> 1506735 T2949 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/'
   [junit4]   2> 1506787 T2949 oasc.CoreContainer.<init> New CoreContainer 742765009
   [junit4]   2> 1506787 T2949 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/]
   [junit4]   2> 1506788 T2949 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
   [junit4]   2> 1506788 T2949 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
   [junit4]   2> 1506788 T2949 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
   [junit4]   2> 1506789 T2949 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1506789 T2949 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 1506789 T2949 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1506789 T2949 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1506789 T2949 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1506789 T2949 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1506790 T2949 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1506792 T2949 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=30000&connTimeout=30000&retry=false
   [junit4]   2> 1506794 T2949 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1506794 T2949 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1506794 T2949 oasc.CoreContainer.load Host Name: 
   [junit4]   2> 1506799 T2950 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 1506800 T2950 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 1506800 T2950 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 1506801 T2950 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 1506857 T2950 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 1506914 T2950 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-spellcheckcomponent.xml
   [junit4]   2> 1506914 T2950 oass.IndexSchema.readSchema Reading Solr Schema from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml
   [junit4]   2> 1506932 T2950 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 1507250 T2950 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1507256 T2950 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1507258 T2950 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1507268 T2950 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1507271 T2950 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1507274 T2950 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1507274 T2950 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1507274 T2950 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1507274 T2950 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1507275 T2950 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1507275 T2950 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1507275 T2950 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from instancedir /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/
   [junit4]   2> 1507275 T2950 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1507276 T2950 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.spelling.SpellCheckCollatorTest-1D09958961FE53F3-001/init-core-data-001/
   [junit4]   2> 1507276 T2950 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
   [junit4]   2> 1507276 T2950 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.spelling.SpellCheckCollatorTest-1D09958961FE53F3-001/init-core-data-001
   [junit4]   2> 1507276 T2950 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.spelling.SpellCheckCollatorTest-1D09958961FE53F3-001/init-core-data-001/index/
   [junit4]   2> 1507277 T2950 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.spelling.SpellCheckCollatorTest-1D09958961FE53F3-001/init-core-data-001/index' doesn't exist. Creating new index...
   [junit4]   2> 1507277 T2950 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.spelling.SpellCheckCollatorTest-1D09958961FE53F3-001/init-core-data-001/index
   [junit4]   2> 1507277 T2950 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=19, maxMergedSegmentMB=75.9609375, floorSegmentMB=0.642578125, forceMergeDeletesPctAllowed=29.829035457023057, segmentsPerTier=8.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6175166426499068
   [junit4]   2> 1507278 T2950 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=BaseDirectoryWrapper(RAMDirectory@26bee14c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3db95873),segFN=segments_1,generation=1}
   [junit4]   2> 1507278 T2950 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1507279 T2950 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1507279 T2950 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1507279 T2950 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1507280 T2950 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1507280 T2950 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1507280 T2950 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler
   [junit4]   2> 1507280 T2950 oasc.RequestHandlers.initHandlersFromConfig created /replication: org.apache.solr.handler.ReplicationHandler
   [junit4]   2> 1507280 T2950 oasc.RequestHandlers.initHandlersFromConfig created /get: org.apache.solr.handler.RealTimeGetHandler
   [junit4]   2> 1507281 T2950 oasc.RequestHandlers.initHandlersFromConfig created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler
   [junit4]   2> 1507281 T2950 oasc.RequestHandlers.initHandlersFromConfig created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler
   [junit4]   2> 1507281 T2950 oasc.RequestHandlers.initHandlersFromConfig created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler
   [junit4]   2> 1507282 T2950 oasc.RequestHandlers.initHandlersFromConfig created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler
   [junit4]   2> 1507282 T2950 oasc.RequestHandlers.initHandlersFromConfig created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler
   [junit4]   2> 1507282 T2950 oasc.RequestHandlers.initHandlersFromConfig created /admin/properties: org.apache.solr.handler.admin.PropertiesRequestHandler
   [junit4]   2> 1507282 T2950 oasc.RequestHandlers.initHandlersFromConfig created /admin/logging: org.apache.solr.handler.admin.LoggingHandler
   [junit4]   2> 1507283 T2950 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: org.apache.solr.handler.admin.ShowFileRequestHandler
   [junit4]   2> 1507283 T2950 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: org.apache.solr.handler.PingRequestHandler
   [junit4]   2> 1507283 T2950 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 1507284 T2950 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 1507284 T2950 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 1507284 T2950 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 1507298 T2950 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1507299 T2950 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1507301 T2950 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1507302 T2950 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1507304 T2950 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1507304 T2950 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1507305 T2950 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=36, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.2272991907764933]
   [junit4]   2> 1507305 T2950 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=BaseDirectoryWrapper(RAMDirectory@26bee14c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3db95873),segFN=segments_1,generation=1}
   [junit4]   2> 1507305 T2950 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1507306 T2950 oass.SolrIndexSearcher.<init> Opening Searcher@4ecd7c0e[collection1] main
   [junit4]   2> 1507306 T2950 oasr.ManagedResourceStorage.newStorageIO WARN Cannot write to config directory /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf; switching to use InMemory storage instead.
   [junit4]   2> 1507306 T2950 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 1507307 T2950 oasr.ManagedResourceStorage.load Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 1507307 T2950 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /rest/managed
   [junit4]   2> 1507307 T2950 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using InMemoryStorage
   [junit4]   2> 1507307 T2950 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 1507307 T2950 oashc.SpellCheckComponent.inform Initializing spell checkers
   [junit4]   2> 1507315 T2950 oashc.SpellCheckComponent.inform Registering newSearcher listener for spellchecker: default
   [junit4]   2> 1507321 T2950 oashc.SpellCheckComponent.inform Registering newSearcher listener for spellchecker: default_teststop
   [junit4]   2> 1507412 T2950 oass.DirectSolrSpellChecker.init init: {name=direct,classname=solr.DirectSolrSpellChecker,minQueryLength=3,maxQueryFrequency=100.0,field=teststop}
   [junit4]   2> 1507413 T2950 oass.DirectSolrSpellChecker.init init: {name=direct_lowerfilt,classname=solr.DirectSolrSpellChecker,minQueryLength=3,maxQueryFrequency=100.0,field=lowerfilt}
   [junit4]   2> 1507436 T2950 oashc.SpellCheckComponent.inform Registering newSearcher listener for spellchecker: threshold
   [junit4]   2> 1507436 T2950 oass.DirectSolrSpellChecker.init init: {name=threshold_direct,classname=solr.DirectSolrSpellChecker,field=lowerfilt,spellcheckIndexDir=spellcheckerThreshold,buildOnCommit=true,thresholdTokenFrequency=0.29}
   [junit4]   2> 1507436 T2950 oashc.SpellCheckComponent.inform Registering newSearcher listener for spellchecker: threshold_direct
   [junit4]   2> 1507442 T2950 oashc.SpellCheckComponent.inform Registering newSearcher listener for spellchecker: multipleFields
   [junit4]   2> 1507486 T2950 oashc.SpellCheckComponent.inform Registering newSearcher listener for spellchecker: freq
   [junit4]   2> 1507500 T2950 oashc.SpellCheckComponent.inform Registering newSearcher listener for spellchecker: fqcn
   [junit4]   2> 1507501 T2950 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 1507501 T2951 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4ecd7c0e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1507501 T2950 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 1507504 T2949 oas.SolrTestCaseJ4.initCore ####initCore end
   [junit4]   2> ASYNC  NEW_CORE C594 name=collection1 org.apache.solr.core.SolrCore@3f0e4d57
   [junit4]   2> 1507506 T2949 C594 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[0]} 0 2
   [junit4]   2> 1507531 T2949 C594 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[1]} 0 3
   [junit4]   2> 1507533 T2949 C594 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[2]} 0 0
   [junit4]   2> 1507536 T2949 C594 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[3]} 0 1
   [junit4]   2> 1507539 T2949 C594 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[4]} 0 1
   [junit4]   2> 1507542 T2949 C594 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[5]} 0 2
   [junit4]   2> 1507544 T2949 C594 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[6]} 0 1
   [junit4]   2> 1507547 T2949 C594 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[7]} 0 2
   [junit4]   2> 1507549 T2949 C594 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[8]} 0 1
   [junit4]   2> 1507552 T2949 C594 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[9]} 0 2
   [junit4]   2> 1507554 T2949 C594 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[10]} 0 1
   [junit4]   2> 1507556 T2949 C594 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[11]} 0 1
   [junit4]   2> 1507558 T2949 C594 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[12]} 0 0
   [junit4]   2> 1507561 T2949 C594 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[13]} 0 1
   [junit4]   2> 1507563 T2949 C594 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[14]} 0 1
   [junit4]   2> 1507566 T2949 C594 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[15]} 0 2
   [junit4]   2> 1507568 T2949 C594 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[16]} 0 1
   [junit4]   2> 1507569 T2949 C594 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1507575 T2949 C594 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=BaseDirectoryWrapper(RAMDirectory@26bee14c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3db95873),segFN=segments_1,generation=1}
   [junit4]   2> 		commit{dir=BaseDirectoryWrapper(RAMDirectory@26bee14c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3db95873),segFN=segments_2,generation=2}
   [junit4]   2> 1507575 T2949 C594 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
   [junit4]   2> 1507577 T2949 C594 oass.SolrIndexSearcher.<init> Opening Searcher@7e89b458[collection1] main
   [junit4]   2> 1507578 T2949 C594 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 1507578 T2951 oashc.SpellCheckComponent$SpellCheckerListener.buildSpellIndex Building spell index for spell checker: default
   [junit4]   2> 1507658 T2951 oashc.SpellCheckComponent$SpellCheckerListener.buildSpellIndex Building spell index for spell checker: default_teststop
   [junit4]   2> 1507712 T2951 oashc.SpellCheckComponent$SpellCheckerListener.buildSpellIndex Building spell index for spell checker: threshold
   [junit4]   2> 1507723 T2951 oashc.SpellCheckComponent$SpellCheckerListener.buildSpellIndex Building spell index for spell checker: threshold_direct
   [junit4]   2> 1507723 T2951 oashc.SpellCheckComponent$SpellCheckerListener.buildSpellIndex Building spell index for spell checker: multipleFields
   [junit4]   2> 1507773 T2951 oashc.SpellCheckComponent$SpellCheckerListener.buildSpellIndex Building spell index for spell checker: freq
   [junit4]   2> 1507816 T2951 oashc.SpellCheckComponent$SpellCheckerListener.buildSpellIndex Building spell index for spell checker: fqcn
   [junit4]   2> 1507853 T2951 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7e89b458[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(5.0.0):C2) Uninverting(_1(5.0.0):C2) Uninverting(_2(5.0.0):C2) Uninverting(_3(5.0.0):C2) Uninverting(_4(5.0.0):C2) Uninverting(_5(5.0.0):C2) Uninverting(_6(5.0.0):C2) Uninverting(_7(5.0.0):C2) Uninverting(_8(5.0.0):C1)))}
   [junit4]   2> 1507853 T2949 C594 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {commit=} 0 284
   [junit4]   2> 1507861 T2949 oas.SolrTestCaseJ4.setUp ###Starting testContextSensitiveCollate
   [junit4]   2> 1508218 T2949 C594 oasc.SolrCore.execute [collection1] webapp=null path=null params={spellcheck.maxResultsForSuggest=0&spellcheck.collate=true&spellcheck.maxCollations=1&spellcheck.extendedResults=true&spellcheck.alternativeTermCount=5&spellcheck=true&wt=xml&spellcheck.dictionary=direct&spellcheck.collateExtendedResults=true&q=teststop:(flew+AND+form+AND+heathrow)&indent=true&spellcheck.build=true&qt=spellCheckCompRH&spellcheck.count=10&spellcheck.maxCollationTries=10} hits=0 status=0 QTime=357 
   [junit4]   2> ASYNC  NEW_CORE C595 name=collection1 org.apache.solr.core.SolrCore@3f0e4d57
   [junit4]   2> 1508273 T2949 C595 oasc.SolrCore.execute [collection1] webapp=null path=null params={spellcheck.maxResultsForSuggest=1&spellcheck.collate=true&spellcheck.maxCollations=1&spellcheck.extendedResults=true&spellcheck.alternativeTermCount=5&spellcheck=true&wt=xml&spellcheck.dictionary=direct&spellcheck.collateExtendedResults=true&q=teststop:(june+AND+customs)&indent=true&qt=spellCheckCompRH&spellcheck.count=10&spellcheck.maxCollationTries=10} hits=1 status=0 QTime=33 
   [junit4]   2> 1508287 T2949 C595 oasc.SolrCore.execute [collection1] webapp=null path=null params={spellcheck.dictionary=direct&mm=2&spellcheck.collate=true&q=teststop:(june+customs)&indent=true&qt=spellCheckCompRH&spellcheck.alternativeTermCount=0&spellcheck=true&wt=xml&spellcheck.count=10} hits=2 status=0 QTime=3 
   [junit4]   2> 1508406 T2949 C595 oasc.SolrCore.execute [collection1] webapp=null path=null params={spellcheck.maxResultsForSuggest=0&spellcheck.collate=true&spellcheck.maxCollations=1&spellcheck.extendedResults=true&spellcheck.alternativeTermCount=5&spellcheck=true&wt=xml&spellcheck.dictionary=default_teststop&spellcheck.collateExtendedResults=true&q=teststop:(flew+AND+form+AND+heathrow)&indent=true&spellcheck.build=true&qt=spellCheckCompRH&spellcheck.count=10&spellcheck.maxCollationTries=10} hits=0 status=0 QTime=118 
   [junit4]   2> 1508425 T2949 C595 oasc.SolrCore.execute [collection1] webapp=null path=null params={spellcheck.maxResultsForSuggest=1&spellcheck.collate=true&spellcheck.maxCollations=1&spellcheck.extendedResults=true&spellcheck.alternativeTermCount=5&spellcheck=true&wt=xml&spellcheck.dictionary=default_teststop&spellcheck.collateExtendedResults=true&q=teststop:(june+AND+customs)&indent=true&qt=spellCheckCompRH&spellcheck.count=10&spellcheck.maxCollationTries=10} hits=1 status=0 QTime=8 
   [junit4]   2> 1508437 T2949 C595 oasc.SolrCore.execute [collection1] webapp=null path=null params={spellcheck.dictionary=default_teststop&mm=2&spellcheck.collate=true&q=teststop:(june+customs)&indent=true&qt=spellCheckCompRH&spellcheck.alternativeTermCount=0&spellcheck=true&wt=xml&spellcheck.count=10} hits=2 status=0 QTime=3 
   [junit4]   2> 1508438 T2949 oas.SolrTestCaseJ4.tearDown ###Ending testContextSensitiveCollate
   [junit4]   2> 1508445 T2949 oas.SolrTestCaseJ4.setUp ###Starting testCollateWithMultipleRequestHandlers
   [junit4]   2> 1508558 T2949 oas.SolrTestCaseJ4.tearDown ###Ending testCollateWithMultipleRequestHandlers
   [junit4]   2> 1508567 T2949 oas.SolrTestCaseJ4.setUp ###Starting testCollateWithGrouping
   [junit4]   2> 1508735 T2949 oas.SolrTestCaseJ4.tearDown ###Ending testCollateWithGrouping
   [junit4]   2> 1508743 T2949 oas.SolrTestCaseJ4.setUp ###Starting testCollateWithOverride
   [junit4]   2> 1508857 T2949 C595 oasc.SolrCore.execute [collection1] webapp=null path=null params={defType=edismax&spellcheck.dictionary=direct&mm=1&qf=teststop&q=partisian+politcal+mashine&spellcheck.collate=true&spellcheck.maxCollations=10&qt=spellCheckCompRH&spellcheck=true&wt=xml&spellcheck.maxCollationTries=10&spellcheck.count=10} hits=0 status=0 QTime=114 
   [junit4]   2> 1508898 T2949 C595 oasc.SolrCore.execute [collection1] webapp=null path=null params={qf=teststop&spellcheck.collate=true&spellcheck.maxCollations=10&spellcheck=true&wt=xml&spellcheck.dictionary=direct&defType=edismax&mm=1&q=partisian+politcal+mashine&spellcheck.collateParam.mm=100%25&qt=spellCheckCompRH&spellcheck.count=10&spellcheck.maxCollationTries=10} hits=0 status=0 QTime=33 
   [junit4]   2> 1508902 T2949 oas.SolrTestCaseJ4.tearDown ###Ending testCollateWithOverride
   [junit4]   2> 1508910 T2949 oas.SolrTestCaseJ4.setUp ###Starting testCollateWithFilter
   [junit4]   2> 1509061 T2949 oas.SolrTestCaseJ4.tearDown ###Ending testCollateWithFilter
   [junit4]   2> 1509067 T2949 oas.SolrTestCaseJ4.setUp ###Starting testZeroTries
   [junit4]   2> 1509184 T2949 oas.SolrTestCaseJ4.tearDown ###Ending testZeroTries
   [junit4]   2> 1509192 T2949 oas.SolrTestCaseJ4.setUp ###Starting testCollationWithHypens
   [junit4]   2> 1509304 T2949 oas.SolrTestCaseJ4.tearDown ###Ending testCollationWithHypens
   [junit4]   2> 1509311 T2949 oas.SolrTestCaseJ4.setUp ###Starting testEstimatedHitCounts
   [junit4]   2> 1509320 T2949 C595 oasc.SolrCore.execute [collection1] webapp=null path=null params={spellcheck=true&spellcheck.dictionary=direct&spellcheck.count=1&spellcheck.collate=true&spellcheck.maxCollationTries=1&spellcheck.maxCollations=1&spellcheck.collateExtendedResults=true&qt=spellCheckCompRH&q=teststop%3Ametnoia} hits=0 status=0 QTime=8 
   [junit4]   2> 1509332 T2949 C595 oasc.SolrCore.execute [collection1] webapp=null path=null params={spellcheck=true&spellcheck.dictionary=direct&spellcheck.count=1&spellcheck.collate=true&spellcheck.maxCollationTries=1&spellcheck.maxCollations=1&spellcheck.collateExtendedResults=true&qt=spellCheckCompRH&q=teststop%3Ametnoia&spellcheck.collateMaxCollectDocs=0} hits=0 status=0 QTime=8 
   [junit4]   2> 1509344 T2949 C595 oasc.SolrCore.execute [collection1] webapp=null path=null params={spellcheck=true&spellcheck.dictionary=direct&spellcheck.count=1&spellcheck.collate=true&spellcheck.maxCollationTries=1&spellcheck.maxCollations=1&spellcheck.collateExtendedResults=true&qt=spellCheckCompRH&q=teststop%3Ametnoia&spellcheck.collateMaxCollectDocs=30} hits=0 status=0 QTime=8 
   [junit4]   2> 1509356 T2949 C595 oasc.SolrCore.execute [collection1] webapp=null path=null params={spellcheck=true&spellcheck.dictionary=direct&spellcheck.count=1&spellcheck.collate=true&spellcheck.maxCollationTries=1&spellcheck.maxCollations=1&spellcheck.collateExtendedResults=true&qt=spellCheckCompRH&q=teststop%3Ametnoia&spellcheck.collateMaxCollectDocs=100} hits=0 status=0 QTime=8 
   [junit4]   2> 1509367 T2949 C595 oasc.SolrCore.execute [collection1] webapp=null path=null params={spellcheck=true&spellcheck.dictionary=direct&spellcheck.count=1&spellcheck.collate=true&spellcheck.maxCollationTries=1&spellcheck.maxCollations=1&spellcheck.collateExtendedResults=true&qt=spellCheckCompRH&q=teststop%3Ametnoia&spellcheck.collateMaxCollectDocs=10000} hits=0 status=0 QTime=7 
   [junit4]   2> 1509379 T2949 C595 oasc.SolrCore.execute [collection1] webapp=null path=null params={spellcheck=true&spellcheck.dictionary=direct&spellcheck.count=1&spellcheck.collate=true&spellcheck.maxCollationTries=1&spellcheck.maxCollations=1&spellcheck.collateExtendedResults=true&qt=spellCheckCompRH&q=teststop%3Ametnoia&spellcheck.collateMaxCollectDocs=11} hits=0 status=0 QTime=8 
   [junit4]   2> 1509391 T2949 C595 oasc.SolrCore.execute [collection1] webapp=null path=null params={spellcheck=true&spellcheck.dictionary=direct&spellcheck.count=1&spellcheck.collate=true&spellcheck.maxCollationTries=1&spellcheck.maxCollations=1&spellcheck.collateExtendedResults=true&qt=spellCheckCompRH&q=teststop%3Ametnoia&spellcheck.collateMaxCollectDocs=17} hits=0 status=0 QTime=8 
   [junit4]   2> 1509403 T2949 C595 oasc.SolrCore.execute [collection1] webapp=null path=null params={spellcheck=true&spellcheck.dictionary=direct&spellcheck.count=1&spellcheck.collate=true&spellcheck.maxCollationTries=1&spellcheck.maxCollations=1&spellcheck.collateExtendedResults=true&qt=spellCheckCompRH&q=teststop%3Ametnoia&spellcheck.collateMaxCollectDocs=9} hits=0 status=0 QTime=7 
   [junit4]   2> 1509415 T2949 C595 oasc.SolrCore.execute [collection1] webapp=null path=null params={spellcheck=true&spellcheck.dictionary=direct&spellcheck.count=1&spellcheck.collate=true&spellcheck.maxCollationTries=1&spellcheck.maxCollations=1&spellcheck.collateExtendedResults=true&qt=spellCheckCompRH&q=teststop%3Ametnoia&spellcheck.collateMaxCollectDocs=8} hits=0 status=0 QTime=8 
   [junit4]   2> 1509428 T2949 C595 oasc.SolrCore.execute [collection1] webapp=null path=null params={spellcheck=true&spellcheck.dictionary=direct&spellcheck.count=1&spellcheck.collate=true&spellcheck.maxCollationTries=1&spellcheck.maxCollations=1&spellcheck.collateExtendedResults=true&qt=spellCheckCompRH&q=teststop%3Ametnoia&spellcheck.collateMaxCollectDocs=12} hits=0 status=0 QTime=8 
   [junit4]   2> 1509441 T2949 C595 oasc.SolrCore.execute [collection1] webapp=null path=null params={spellcheck=true&spellcheck.dictionary=direct&spellcheck.count=1&spellcheck.collate=true&spellcheck.maxCollationTries=1&spellcheck.maxCollations=1&spellcheck.collateExtendedResults=true&qt=spellCheckCompRH&q=teststop%3Ametnoia&spellcheck.collateMaxCollectDocs=7} hits=0 status=0 QTime=9 
   [junit4]   2> 1509452 T2949 C595 oasc.SolrCore.execute [collection1] webapp=null path=null params={spellcheck=true&spellcheck.dictionary=direct&spellcheck.count=1&spellcheck.collate=true&spellcheck.maxCollationTries=1&spellcheck.maxCollations=1&spellcheck.collateExtendedResults=true&qt=spellCheckCompRH&q=teststop%3Ametnoia&spellcheck.collateMaxCollectDocs=15} hits=0 status=0 QTime=7 
   [junit4]   2> 1509463 T2949 C595 oasc.SolrCore.execute [collection1] webapp=null path=null params={spellcheck=true&spellcheck

[...truncated too long message...]

rg.apache.solr.core.SolrCores.close(SolrCores.java:117)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer.shutdown(CoreContainer.java:365)
   [junit4]   2> 	at org.apache.solr.util.TestHarness.close(TestHarness.java:337)
   [junit4]   2> 	at org.apache.solr.SolrTestCaseJ4.deleteCore(SolrTestCaseJ4.java:640)
   [junit4]   2> 	at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:185)
   [junit4]   2> 	at sun.reflect.GeneratedMethodAccessor33.invoke(Unknown Source)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:790)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 2119006 T12092 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 2119007 T12092 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.highlight.HighlighterTest-1D09958961FE53F3-001/init-core-data-001 [CachedDir<<refCount=0;path=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.highlight.HighlighterTest-1D09958961FE53F3-001/init-core-data-001;done=false>>]
   [junit4]   2> 2119007 T12092 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.highlight.HighlighterTest-1D09958961FE53F3-001/init-core-data-001
   [junit4]   2> 2119007 T12092 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.highlight.HighlighterTest-1D09958961FE53F3-001/init-core-data-001/index [CachedDir<<refCount=0;path=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.highlight.HighlighterTest-1D09958961FE53F3-001/init-core-data-001/index;done=false>>]
   [junit4]   2> 2119008 T12092 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.highlight.HighlighterTest-1D09958961FE53F3-001/init-core-data-001/index
   [junit4]   2> NOTE: leaving temporary files on disk at: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.highlight.HighlighterTest-1D09958961FE53F3-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene50): {title_stringNoNorms=PostingsFormat(name=LuceneVarGapDocFreqInterval), title_stemmed=PostingsFormat(name=Memory doPackFST= true), foo_t=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), text=BlockTreeOrds(blocksize=128), multiDefault=PostingsFormat(name=Memory doPackFST= true), range_facet_l=PostingsFormat(name=LuceneVarGapDocFreqInterval), tv_text=PostingsFormat(name=LuceneVarGapDocFreqInterval), t_text1=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), title_lettertok=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), t_text2=PostingsFormat(name=LuceneVarGapDocFreqInterval), timestamp=PostingsFormat(name=LuceneVarGapDocFreqInterval), title=PostingsFormat(name=Memory doPackFST= true), intDefault=PostingsFormat(name=LuceneVarGapDocFreqInterval), subword_offsets=BlockTreeOrds(blocksize=128), tv_mv_text=PostingsFormat(name=LuceneVarGapDocFreqInterval), id=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), tv_no_off_text=PostingsFormat(name=LuceneVarGapDocFreqInterval), foo_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), t_text=PostingsFormat(name=Memory doPackFST= true), textgap=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), subword=PostingsFormat(name=LuceneVarGapDocFreqInterval), cat=PostingsFormat(name=LuceneVarGapDocFreqInterval), weight=PostingsFormat(name=Memory doPackFST= true), _version_=PostingsFormat(name=LuceneVarGapDocFreqInterval), foo_sI=BlockTreeOrds(blocksize=128)}, docValues:{timestamp=DocValuesFormat(name=Direct)}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=sq_AL, timezone=Kwajalein
   [junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_65 (64-bit)/cpus=16,threads=1,free=244415520,total=376438784
   [junit4]   2> NOTE: All tests run in this JVM: [TestFastOutputStream, SolrCoreCheckLockOnStartupTest, TestAtomicUpdateErrorCases, SortByFunctionTest, TestComplexPhraseQParserPlugin, AutoCommitTest, NoCacheHeaderTest, SystemInfoHandlerTest, TestBM25SimilarityFactory, SolrCoreTest, PeerSyncTest, TestExceedMaxTermLength, TestPartialUpdateDeduplication, HdfsDirectoryTest, BJQParserTest, TestReRankQParserPlugin, DirectUpdateHandlerOptimizeTest, TestSolrXml, TestFunctionQuery, SolrXmlInZkTest, CloudExitableDirectoryReaderTest, SharedFSAutoReplicaFailoverUtilsTest, TestDocumentBuilder, CoreAdminRequestStatusTest, DeleteShardTest, BasicZkTest, MBeansHandlerTest, TestDynamicFieldResource, TestReload, TestGroupingSearch, CoreMergeIndexesAdminHandlerTest, CurrencyFieldOpenExchangeTest, DistributedIntervalFacetingTest, TestLFUCache, TestNonDefinedSimilarityFactory, DeleteInactiveReplicaTest, TestCloudManagedSchemaConcurrent, DistributedFacetPivotLargeTest, CachingDirectoryFactoryTest, DistributedQueueTest, TestDefaultSearchFieldResource, SolrTestCaseJ4Test, RequiredFieldsTest, CustomCollectionTest, StressHdfsTest, DistributedSuggestComponentTest, HighlighterMaxOffsetTest, TestBinaryField, RankQueryTest, URLClassifyProcessorTest, TestSolrQueryParserDefaultOperatorResource, SuggestComponentTest, DebugComponentTest, UnloadDistributedZkTest, SyncSliceTest, TestZkChroot, ZkCLITest, ShardRoutingCustomTest, TestDistributedSearch, SpellCheckComponentTest, ConvertedLegacyTest, OverseerCollectionProcessorTest, HighlighterTest]
   [junit4] Completed on J1 in 5.23s, 31 tests, 9 errors <<< FAILURES!

[...truncated 722 lines...]
BUILD FAILED
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:532: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:473: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:61: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/extra-targets.xml:39: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:508: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1363: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:970: There were test failures: 450 suites, 1841 tests, 10 errors, 56 ignored (10 assumptions)

Total time: 187 minutes 27 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #680
Archived 5 artifacts
Archive block size is 32768
Received 0 blocks and 686712727 bytes
Compression is 0.0%
Took 2 min 6 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 687 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/687/

9 tests failed.
FAILED:  org.apache.solr.highlight.HighlighterTest.testTermVecMultiValuedHighlight

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([918F00AE3D901920:3BD104C2887A0D38]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:729)
	at org.apache.solr.highlight.HighlighterTest.testTermVecMultiValuedHighlight(HighlighterTest.java:211)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//lst[@name='1']/arr[@name='tv_mv_text']/str[.='a <em>long</em> days night this should be a piece of text which']
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">350</int></lst><result name="response" numFound="1" start="0"><doc><arr name="tv_mv_text"><str>a long days night this should be a piece of text which is is is is is is is is is is is is is is is is is is is is is is is is isis is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is sufficiently lengthly to produce multiple fragments which are not concatenated at all--we want two disjoint long fragments.</str><str>a long days night this should be a piece of text which is is is is is is is is is is is is is is is is is is is is is is is is isis is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is sufficiently lengthly to produce multiple fragments which are not concatenated at all--we want two disjoint long fragments.</str></arr><int name="id">1</int><arr name="range_facet_l"><long>1</long></arr><long name="_version_">1486045057872035840</long><int name="intDefault">42</int><arr name="multiDefault"><str>muLti-Default</str></arr><date name="timestamp">2014-11-28T19:30:04.715Z</date></doc></result><lst name="highlighting"><lst name="1"/></lst>
</response>

	request was:hl.fl=tv_mv_text&q=tv_mv_text:long&hl=true&rows=200&qt=standard&start=0&hl.snippets=2
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:722)
	... 41 more


FAILED:  org.apache.solr.highlight.HighlighterTest.testVariableFragsize

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([918F00AE3D901920:96C300DFDF2EB5B2]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:729)
	at org.apache.solr.highlight.HighlighterTest.testVariableFragsize(HighlighterTest.java:592)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//lst[@name='1']/arr[@name='tv_text']/str[.='a <em>long</em> days night this should be a piece of text which']
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">2</int></lst><result name="response" numFound="1" start="0"><doc><arr name="tv_text"><str>a long days night this should be a piece of text which is is is is is is is is is is is is is is is is is is is is is is is is isis is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is sufficiently lengthly to produce multiple fragments which are not concatenated at all</str></arr><int name="id">1</int><arr name="range_facet_l"><long>1</long></arr><long name="_version_">1486045058438266880</long><int name="intDefault">42</int><arr name="multiDefault"><str>muLti-Default</str></arr><date name="timestamp">2014-11-28T19:30:05.255Z</date></doc></result><lst name="highlighting"><lst name="1"/></lst>
</response>

	request was:hl.fl=tv_text&q=tv_text:long&hl=true&rows=200&qt=standard&start=0
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:722)
	... 41 more


FAILED:  org.apache.solr.highlight.HighlighterTest.testSubwordWildcardHighlightWithTermOffsets2

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([918F00AE3D901920:A32C5B27FD3CE34E]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:729)
	at org.apache.solr.highlight.HighlighterTest.testSubwordWildcardHighlightWithTermOffsets2(HighlighterTest.java:843)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//lst[@name='highlighting']/lst[@name='1']/arr[@name='subword_offsets']/str='lorem <em>PowerShot</em> ipsum'
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">128</int></lst><result name="response" numFound="1" start="0"><doc><arr name="subword_offsets"><str>lorem PowerShot ipsum</str></arr><int name="id">1</int><arr name="range_facet_l"><long>1</long></arr><long name="_version_">1486045058677342208</long><int name="intDefault">42</int><arr name="multiDefault"><str>muLti-Default</str></arr><date name="timestamp">2014-11-28T19:30:05.483Z</date></doc></result><lst name="highlighting"><lst name="1"/></lst>
</response>

	request was:hl.fl=subword_offsets&hl=true&q=subword_offsets:pow*&wt=xml
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:722)
	... 41 more


FAILED:  org.apache.solr.highlight.HighlighterTest.testTermVecMultiValuedHighlight2

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([918F00AE3D901920:89063D72A3438C12]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:729)
	at org.apache.solr.highlight.HighlighterTest.testTermVecMultiValuedHighlight2(HighlighterTest.java:239)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//lst[@name='1']/arr[@name='tv_mv_text']/str[.='a <em>long</em> days night this should be a piece of text which']
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">2</int></lst><result name="response" numFound="1" start="0"><doc><arr name="tv_mv_text"><str>short</str><str>a long days night this should be a piece of text which is is is is is is is is is is is is is is is is is is is is is is is is isis is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is sufficiently lengthly to produce multiple fragments which are not concatenated at all--we want two disjoint long fragments.</str></arr><int name="id">1</int><arr name="range_facet_l"><long>1</long></arr><long name="_version_">1486045059092578304</long><int name="intDefault">42</int><arr name="multiDefault"><str>muLti-Default</str></arr><date name="timestamp">2014-11-28T19:30:05.879Z</date></doc></result><lst name="highlighting"><lst name="1"/></lst>
</response>

	request was:hl.fl=tv_mv_text&q=tv_mv_text:long&hl=true&rows=200&qt=standard&start=0&hl.snippets=2
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:722)
	... 41 more


FAILED:  org.apache.solr.highlight.HighlighterTest.testDisMaxHighlight

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([918F00AE3D901920:8B272F2A7C4EE8B1]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:729)
	at org.apache.solr.highlight.HighlighterTest.testDisMaxHighlight(HighlighterTest.java:262)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//lst[@name='1']/arr[@name='tv_text']/str
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">117</int></lst><result name="response" numFound="1" start="0"><doc><arr name="tv_text"><str>a long day's night</str></arr><int name="id">1</int><arr name="range_facet_l"><long>1</long></arr><long name="_version_">1486045059150249984</long><int name="intDefault">42</int><arr name="multiDefault"><str>muLti-Default</str></arr><date name="timestamp">2014-11-28T19:30:05.934Z</date></doc></result><lst name="highlighting"><lst name="1"/></lst>
</response>

	request was:qf=tv_text&hl.fl=tv_text&q=long&hl=true&rows=200&qt=dismax&q.alt=*:*&start=0
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:722)
	... 41 more


FAILED:  org.apache.solr.highlight.HighlighterTest.testTermVecHighlight

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([918F00AE3D901920:B0AF787F34C3CD33]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:729)
	at org.apache.solr.highlight.HighlighterTest.testTermVecHighlight(HighlighterTest.java:145)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//lst[@name='1']/arr[@name='tv_text']/str[.='a <em>long</em> days night this should be a piece of text which']
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">2</int></lst><result name="response" numFound="1" start="0"><doc><arr name="tv_text"><str>a long days night this should be a piece of text which is is is is is is is is is is is is is is is is is is is is is is is is isis is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is sufficiently lengthly to produce multiple fragments which are not concatenated at all--we want two disjoint long fragments.</str></arr><int name="id">1</int><arr name="range_facet_l"><long>1</long></arr><long name="_version_">1486045059374645248</long><int name="intDefault">42</int><arr name="multiDefault"><str>muLti-Default</str></arr><date name="timestamp">2014-11-28T19:30:06.148Z</date></doc></result><lst name="highlighting"><lst name="1"/></lst>
</response>

	request was:hl.fl=tv_text&q=tv_text:long&hl=true&rows=200&qt=standard&start=0&hl.snippets=2
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:722)
	... 41 more


FAILED:  org.apache.solr.highlight.HighlighterTest.testSubwordWildcardHighlightWithTermOffsets

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([918F00AE3D901920:AE29E46DC8357B5D]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:729)
	at org.apache.solr.highlight.HighlighterTest.testSubwordWildcardHighlightWithTermOffsets(HighlighterTest.java:834)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//lst[@name='highlighting']/lst[@name='1']/arr[@name='subword_offsets']/str='lorem <em>PowerShot.com</em> ipsum'
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">2</int></lst><result name="response" numFound="1" start="0"><doc><arr name="subword_offsets"><str>lorem PowerShot.com ipsum</str></arr><int name="id">1</int><arr name="range_facet_l"><long>1</long></arr><long name="_version_">1486045059535077376</long><int name="intDefault">42</int><arr name="multiDefault"><str>muLti-Default</str></arr><date name="timestamp">2014-11-28T19:30:06.301Z</date></doc></result><lst name="highlighting"><lst name="1"/></lst>
</response>

	request was:hl.fl=subword_offsets&hl=true&q=subword_offsets:pow*&wt=xml
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:722)
	... 41 more


FAILED:  org.apache.solr.highlight.HighlighterTest.testTwoFieldHighlight

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([918F00AE3D901920:E8FDC5ACDAE64F9A]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:729)
	at org.apache.solr.highlight.HighlighterTest.testTwoFieldHighlight(HighlighterTest.java:391)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//lst[@name='1']/arr[@name='tv_text']/str
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">1</int></lst><result name="response" numFound="1" start="0"><doc><arr name="t_text"><str>a long day's night</str></arr><int name="id">1</int><arr name="range_facet_l"><long>1</long></arr><arr name="tv_text"><str>a long night's day</str></arr><long name="_version_">1486045059713335296</long><int name="intDefault">42</int><arr name="multiDefault"><str>muLti-Default</str></arr><date name="timestamp">2014-11-28T19:30:06.471Z</date></doc></result><lst name="highlighting"><lst name="1"><arr name="t_text"><str>a &lt;em&gt;long&lt;/em&gt; day's night</str></arr></lst></lst>
</response>

	request was:hl.fl=t_text+tv_text&q=t_text:long&hl=true&rows=200&qt=standard&start=0
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:722)
	... 41 more


FAILED:  org.apache.solr.highlight.HighlighterTest.testLongFragment

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([918F00AE3D901920:9487C0932BC12E8]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:729)
	at org.apache.solr.highlight.HighlighterTest.testLongFragment(HighlighterTest.java:496)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//lst[@name='1']/arr[@name='tv_text']/str
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">2</int></lst><result name="response" numFound="1" start="0"><doc><arr name="tv_text"><str>junit: [mkdir] Created dir: /home/klaas/worio/backend/trunk/build-src/solr-nightly/build/test-results [junit] Running org.apache.solr.BasicFunctionalityTest [junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed: 5.36 sec [junit] Running org.apache.solr.ConvertedLegacyTest [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 8.268 sec [junit] Running org.apache.solr.DisMaxRequestHandlerTest [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 1.56 sec [junit] Running org.apache.solr.HighlighterTest [junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed: 4.979 sec [junit] Running org.apache.solr.OutputWriterTest [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.797 sec [junit] Running org.apache.solr.SampleTest [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 1.021 sec [junit] Running org.apache.solr.analysis.TestBufferedTokenStream [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.05 sec [junit] Running org.apache.solr.analysis.TestRemoveDuplicatesTokenFilter [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.054 sec [junit] Running org.apache.solr.analysis.TestSynonymFilter [junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 0.081 sec [junit] Running org.apache.solr.analysis.TestWordDelimiterFilter [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 1.714 sec [junit] Running org.apache.solr.search.TestDocSet [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.788 sec [junit] Running org.apache.solr.util.SolrPluginUtilsTest [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 3.519 sec [junit] Running org.apache.solr.util.TestOpenBitSet [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.533 sec</str></arr><int name="id">1</int><arr name="range_facet_l"><long>1</long></arr><long name="_version_">1486045059922001920</long><int name="intDefault">42</int><arr name="multiDefault"><str>muLti-Default</str></arr><date name="timestamp">2014-11-28T19:30:06.67Z</date></doc></result><lst name="highlighting"><lst name="1"/></lst>
</response>

	request was:hl.fl=tv_text&q=tv_text:dir&hl=true&rows=200&qt=standard&start=0
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:722)
	... 41 more




Build Log:
[...truncated 10128 lines...]
   [junit4] Suite: org.apache.solr.highlight.HighlighterTest
   [junit4]   2> Creating dataDir: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.highlight.HighlighterTest-918F00AE3D901920-001/init-core-data-001
   [junit4]   2> 1215339 T1481 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (true)
   [junit4]   2> 1215340 T1481 oas.SolrTestCaseJ4.initCore ####initCore
   [junit4]   2> 1215340 T1481 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 1215342 T1481 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 1215342 T1481 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 1215343 T1481 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 1215439 T1481 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 1215505 T1481 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1215506 T1481 oass.IndexSchema.readSchema Reading Solr Schema from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml
   [junit4]   2> 1215524 T1481 oass.IndexSchema.readSchema [null] Schema name=test
   [junit4]   2> 1216019 T1481 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1216033 T1481 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1216036 T1481 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1216053 T1481 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1216058 T1481 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1216062 T1481 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1216062 T1481 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1216063 T1481 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1216063 T1481 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1216064 T1481 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1216064 T1481 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1216064 T1481 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1216064 T1481 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr
   [junit4]   2> 1216065 T1481 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/'
   [junit4]   2> 1216124 T1481 oasc.CoreContainer.<init> New CoreContainer 1162281241
   [junit4]   2> 1216125 T1481 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/]
   [junit4]   2> 1216126 T1481 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
   [junit4]   2> 1216126 T1481 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
   [junit4]   2> 1216126 T1481 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
   [junit4]   2> 1216126 T1481 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1216127 T1481 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 1216127 T1481 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1216127 T1481 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1216127 T1481 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1216128 T1481 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1216128 T1481 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1216131 T1481 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=30000&connTimeout=30000&retry=false
   [junit4]   2> 1216133 T1481 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1216133 T1481 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1216133 T1481 oasc.CoreContainer.load Host Name: 
   [junit4]   2> 1216138 T1482 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 1216139 T1482 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 1216139 T1482 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 1216140 T1482 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 1216200 T1482 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 1216265 T1482 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1216266 T1482 oass.IndexSchema.readSchema Reading Solr Schema from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml
   [junit4]   2> 1216280 T1482 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 1216630 T1482 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1216639 T1482 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1216641 T1482 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1216656 T1482 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1216660 T1482 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1216664 T1482 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1216665 T1482 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1216665 T1482 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1216665 T1482 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1216666 T1482 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1216666 T1482 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1216666 T1482 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from instancedir /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/
   [junit4]   2> 1216666 T1482 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1216667 T1482 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.highlight.HighlighterTest-918F00AE3D901920-001/init-core-data-001/
   [junit4]   2> 1216667 T1482 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2e3f9952
   [junit4]   2> 1216668 T1482 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.highlight.HighlighterTest-918F00AE3D901920-001/init-core-data-001
   [junit4]   2> 1216668 T1482 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.highlight.HighlighterTest-918F00AE3D901920-001/init-core-data-001/index/
   [junit4]   2> 1216668 T1482 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.highlight.HighlighterTest-918F00AE3D901920-001/init-core-data-001/index' doesn't exist. Creating new index...
   [junit4]   2> 1216669 T1482 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.highlight.HighlighterTest-918F00AE3D901920-001/init-core-data-001/index
   [junit4]   2> 1216669 T1482 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=37, maxMergedSegmentMB=66.90234375, floorSegmentMB=1.505859375, forceMergeDeletesPctAllowed=18.809011578117186, segmentsPerTier=12.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.26571195746156095
   [junit4]   2> 1216670 T1482 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@393625f7 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@11f69879)),segFN=segments_1,generation=1}
   [junit4]   2> 1216670 T1482 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1216670 T1482 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
   [junit4]   2> 1216675 T1482 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1216675 T1482 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1216676 T1482 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 1216676 T1482 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 1216676 T1482 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1216677 T1482 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1216677 T1482 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 1216677 T1482 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 1216677 T1482 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1216678 T1482 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1216678 T1482 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1216678 T1482 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1216679 T1482 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1216680 T1482 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1216680 T1482 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1216680 T1482 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1216681 T1482 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler
   [junit4]   2> 1216681 T1482 oasc.RequestHandlers.initHandlersFromConfig created /replication: org.apache.solr.handler.ReplicationHandler
   [junit4]   2> 1216682 T1482 oasc.RequestHandlers.initHandlersFromConfig created /get: org.apache.solr.handler.RealTimeGetHandler
   [junit4]   2> 1216682 T1482 oasc.RequestHandlers.initHandlersFromConfig created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler
   [junit4]   2> 1216683 T1482 oasc.RequestHandlers.initHandlersFromConfig created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler
   [junit4]   2> 1216684 T1482 oasc.RequestHandlers.initHandlersFromConfig created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler
   [junit4]   2> 1216684 T1482 oasc.RequestHandlers.initHandlersFromConfig created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler
   [junit4]   2> 1216685 T1482 oasc.RequestHandlers.initHandlersFromConfig created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler
   [junit4]   2> 1216685 T1482 oasc.RequestHandlers.initHandlersFromConfig created /admin/properties: org.apache.solr.handler.admin.PropertiesRequestHandler
   [junit4]   2> 1216686 T1482 oasc.RequestHandlers.initHandlersFromConfig created /admin/logging: org.apache.solr.handler.admin.LoggingHandler
   [junit4]   2> 1216686 T1482 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: org.apache.solr.handler.PingRequestHandler
   [junit4]   2> 1216687 T1482 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 1216687 T1482 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
   [junit4]   2> 1216687 T1482 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
   [junit4]   2> 1216688 T1482 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 1216688 T1482 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 1216689 T1482 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
   [junit4]   2> 1216689 T1482 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
   [junit4]   2> 1216689 T1482 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
   [junit4]   2> 1216690 T1482 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 1216690 T1482 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 1216691 T1482 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 1216691 T1482 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 1216691 T1482 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 1216692 T1482 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 1216692 T1482 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 1216693 T1482 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 1216693 T1482 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
   [junit4]   2> 1216694 T1482 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
   [junit4]   2> 1216716 T1482 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1216717 T1482 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1216719 T1482 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1216721 T1482 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1216723 T1482 oasc.SolrCore.initDeprecatedSupport WARN solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
   [junit4]   2> 1216727 T1482 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1216727 T1482 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1216728 T1482 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@59253f9d
   [junit4]   2> 1216728 T1482 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@393625f7 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@11f69879)),segFN=segments_1,generation=1}
   [junit4]   2> 1216728 T1482 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1216729 T1482 oass.SolrIndexSearcher.<init> Opening Searcher@13fbc3ba[collection1] main
   [junit4]   2> 1216729 T1482 oasr.RestManager.init Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
   [junit4]   2> 1216730 T1482 oasr.ManagedResourceStorage.load Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 1216730 T1482 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /rest/managed
   [junit4]   2> 1216730 T1482 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using InMemoryStorage
   [junit4]   2> 1216730 T1482 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 1216730 T1482 oashc.SpellCheckComponent.inform Initializing spell checkers
   [junit4]   2> 1216767 T1482 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 1216818 T1482 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 1216820 T1483 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@13fbc3ba[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1216823 T1482 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 1216826 T1481 oas.SolrTestCaseJ4.initCore ####initCore end
   [junit4]   2> 1216833 T1481 oas.SolrTestCaseJ4.setUp ###Starting testTermVecMultiValuedHighlight
   [junit4]   2> ASYNC  NEW_CORE C826 name=collection1 org.apache.solr.core.SolrCore@6ae200e0
   [junit4]   2> 1216841 T1481 C826 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[1 (1486045057872035840)]} 0 7
   [junit4]   2> 1216862 T1481 C826 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1216881 T1481 C826 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@393625f7 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@11f69879)),segFN=segments_1,generation=1}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@393625f7 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@11f69879)),segFN=segments_2,generation=2}
   [junit4]   2> 1216882 T1481 C826 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
   [junit4]   2> 1216915 T1481 C826 oass.SolrIndexSearcher.<init> Opening Searcher@22b2c7d8[collection1] main
   [junit4]   2> 1216916 T1481 C826 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 1216917 T1483 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@22b2c7d8[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(5.0.0):C1)))}
   [junit4]   2> 1216918 T1481 C826 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {commit=} 0 56
   [junit4]   2> 1216920 T1481 C826 oasu.DirectUpdateHandler2.commit start commit{,optimize=true,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1216921 T1481 C826 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 1216921 T1481 C826 oasc.SolrCore.openNewSearcher SolrIndexSearcher has not changed - not re-opening: org.apache.solr.search.SolrIndexSearcher
   [junit4]   2> 1216922 T1481 C826 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 1216922 T1481 C826 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {optimize=} 0 2
   [junit4]   2> 1217274 T1481 C826 oasc.SolrCore.execute [collection1] webapp=null path=null params={hl.fl=tv_mv_text&q=tv_mv_text:long&hl=true&rows=200&qt=standard&start=0&hl.snippets=2} hits=1 status=0 QTime=350 
   [junit4]   2> 1217285 T1481 oas.SolrTestCaseJ4.assertQ ERROR REQUEST FAILED: xpath=//lst[@name='1']/arr[@name='tv_mv_text']/str[.='a <em>long</em> days night this should be a piece of text which']
   [junit4]   2> 		xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]   2> 	<response>
   [junit4]   2> 	<lst name="responseHeader"><int name="status">0</int><int name="QTime">350</int></lst><result name="response" numFound="1" start="0"><doc><arr name="tv_mv_text"><str>a long days night this should be a piece of text which is is is is is is is is is is is is is is is is is is is is is is is is isis is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is sufficiently lengthly to produce multiple fragments which are not concatenated at all--we want two disjoint long fragments.</str><str>a long days night this should be a piece of text which is is is is is is is is is is is is is is is is is is is is is is is is isis is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is sufficiently lengthly to produce multiple fragments which are not concatenated at all--we want two disjoint long fragments.</str></arr><int name="id">1</int><arr name="range_facet_l"><long>1</long></arr><long name="_version_">1486045057872035840</long><int name="intDefault">42</int><arr name="multiDefault"><str>muLti-Default</str></arr><date name="timestamp">2014-11-28T19:30:04.715Z</date></doc></result><lst name="highlighting"><lst name="1"/></lst>
   [junit4]   2> 	</response>
   [junit4]   2> 	
   [junit4]   2> 		request was:hl.fl=tv_mv_text&q=tv_mv_text:long&hl=true&rows=200&qt=standard&start=0&hl.snippets=2
   [junit4]   2> 1217286 T1481 oasc.SolrException.log ERROR REQUEST FAILED: hl.fl=tv_mv_text&q=tv_mv_text:long&hl=true&rows=200&qt=standard&start=0&hl.snippets=2:java.lang.RuntimeException: REQUEST FAILED: xpath=//lst[@name='1']/arr[@name='tv_mv_text']/str[.='a <em>long</em> days night this should be a piece of text which']
   [junit4]   2> 		xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]   2> 	<response>
   [junit4]   2> 	<lst name="responseHeader"><int name="status">0</int><int name="QTime">350</int></lst><result name="response" numFound="1" start="0"><doc><arr name="tv_mv_text"><str>a long days night this should be a piece of text which is is is is is is is is is is is is is is is is is is is is is is is is isis is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is sufficiently lengthly to produce multiple fragments which are not concatenated at all--we want two disjoint long fragments.</str><str>a long days night this should be a piece of text which is is is is is is is is is is is is is is is is is is is is is is is is isis is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is sufficiently lengthly to produce multiple fragments which are not concatenated at all--we want two disjoint long fragments.</str></arr><int name="id">1</int><arr name="range_facet_l"><long>1</long></arr><long name="_version_">1486045057872035840</long><int name="intDefault">42</int><arr name="multiDefault"><str>muLti-Default</str></arr><date name="timestamp">2014-11-28T19:30:04.715Z</date></doc></result><lst name="highlighting"><lst name="1"/></lst>
   [junit4]   2> 	</response>
   [junit4]   2> 	
   [junit4]   2> 		request was:hl.fl=tv_mv_text&q=tv_mv_text:long&hl=true&rows=200&qt=standard&start=0&hl.snippets=2
   [junit4]   2> 		at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:722)
   [junit4]   2> 		at org.apache.solr.highlight.HighlighterTest.testTermVecMultiValuedHighlight(HighlighterTest.java:211)
   [junit4]   2> 		at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 		at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 		at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 		at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2> 		at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2> 		at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	
   [junit4]   2> 1217290 T1481 C826 oass.SolrIndexSearcher.<init> Opening Searcher@300772bf[collection1] realtime
   [junit4]   2> 1217291 T1481 C826 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {deleteByQuery=*:* (-1486045058348089344)} 0 3
   [junit4]   2> 1217293 T1481 oas.SolrTestCaseJ4.tearDown ###Ending testTermVecMultiValuedHighlight
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=HighlighterTest -Dtests.method=testTermVecMultiValuedHighlight -Dtests.seed=918F00AE3D901920 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=de -Dtests.timezone=Mideast/Riyadh87 -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.54s J2 | HighlighterTest.testTermVecMultiValuedHighlight <<<
   [junit4]    > Throwable #1: java.lang.RuntimeException: Exception during query
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([918F00AE3D901920:3BD104C2887A0D38]:0)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:729)
   [junit4]    > 	at org.apache.solr.highlight.HighlighterTest.testTermVecMultiValuedHighlight(HighlighterTest.java:211)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]    > Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//lst[@name='1']/arr[@name='tv_mv_text']/str[.='a <em>long</em> days night this should be a piece of text which']
   [junit4]    > 	xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]    > <response>
   [junit4]    > <lst name="responseHeader"><int name="status">0</int><int name="QTime">350</int></lst><result name="response" numFound="1" start="0"><doc><arr name="tv_mv_text"><str>a long days night this should be a piece of text which is is is is is is is is is is is is is is is is is is is is is is is is isis is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is sufficiently lengthly to produce multiple fragments which are not concatenated at all--we want two disjoint long fragments.</str><str>a long days night this should be a piece of text which is is is is is is is is is is is is is is is is is is is is is is is is isis is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is sufficiently lengthly to produce multiple fragments which are not concatenated at all--we want two disjoint long fragments.</str></arr><int name="id">1</int><arr name="range_facet_l"><long>1</long></arr><long name="_version_">1486045057872035840</long><int name="intDefault">42</int><arr name="multiDefault"><str>muLti-Default</str></arr><date name="timestamp">2014-11-28T19:30:04.715Z</date></doc></result><lst name="highlighting"><lst name="1"/></lst>
   [junit4]    > </response>
   [junit4]    > 	request was:hl.fl=tv_mv_text&q=tv_mv_text:long&hl=true&rows=200&qt=standard&start=0&hl.snippets=2
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:722)
   [junit4]    > 	... 41 more
   [junit4]   2> 1217374 T1481 oas.SolrTestCaseJ4.setUp ###Starting testVariableFragsize
   [junit4]   2> 1217378 T1481 C826 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[1 (1486045058438266880)]} 0 4
   [junit4]   2> 1217380 T1481 C826 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1217390 T1481 C826 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@393625f7 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@11f69879)),segFN=segments_2,generation=2}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@393625f7 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@11f69879)),segFN=segments_3,generation=3}
   [junit4]   2> 1217391 T1481 C826 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
   [junit4]   2> 1217393 T1481 C826 oass.SolrIndexSearcher.<init> Opening Searcher@5e4f5370[collection1] main
   [junit4]   2> 1217394 T1481 C826 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 1217396 T1483 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5e4f5370[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_1(5.0.0):C1)))}
   [junit4]   2> 1217397 T1481 C826 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {commit=} 0 17
   [junit4]   2> 1217400 T1481 C826 oasu.DirectUpdateHandler2.commit start commit{,optimize=true,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1217400 T1481 C826 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 1217401 T1481 C826 oasc.SolrCore.openNewSearcher SolrIndexSearcher has not changed - not re-opening: org.apache.solr.search.SolrIndexSearcher
   [junit4]   2> 1217401 T1481 C826 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 1217402 T1481 C826 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {optimize=} 0 2
   [junit4]   2> 1217405 T1481 C826 oasc.SolrCore.execute [collection1] webapp=null path=null params={hl.fl=tv_text&q=tv_text:long&hl=true&rows=200&qt=standard&start=0} hits=1 status=0 QTime=2 
   [junit4]   2> 1217409 T1481 oas.SolrTestCaseJ4.assertQ ERROR REQUEST FAILED: xpath=//lst[@name='1']/arr[@name='tv_text']/str[.='a <em>long</em> days night this should be a piece of text which']
   [junit4]   2> 		xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]   2> 	<response>
   [junit4]   2> 	<lst name="responseHeader"><int name="status">0</int><int name="QTime">2</int></lst><result name="response" numFound="1" start="0"><doc><arr name="tv_text"><str>a long days night this should be a piece of text which is is is is is is is is is is is is is is is is is is is is is is is is isis is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is suffi

[...truncated too long message...]

rCore.java:1699)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1444)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1379)
   [junit4]   2> 	at org.apache.solr.handler.ReplicationHandler.getIndexVersion(ReplicationHandler.java:558)
   [junit4]   2> 	at org.apache.solr.handler.ReplicationHandler.getStatistics(ReplicationHandler.java:575)
   [junit4]   2> 	at org.apache.solr.core.JmxMonitoredMap$SolrDynamicMBean.getMBeanInfo(JmxMonitoredMap.java:257)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getClassName(DefaultMBeanServerInterceptor.java:1804)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
   [junit4]   2> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
   [junit4]   2> 	at org.apache.solr.core.JmxMonitoredMap.clear(JmxMonitoredMap.java:124)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.close(SolrCore.java:1124)
   [junit4]   2> 	at org.apache.solr.core.SolrCores.close(SolrCores.java:117)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer.shutdown(CoreContainer.java:365)
   [junit4]   2> 	at org.apache.solr.util.TestHarness.close(TestHarness.java:337)
   [junit4]   2> 	at org.apache.solr.SolrTestCaseJ4.deleteCore(SolrTestCaseJ4.java:640)
   [junit4]   2> 	at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:185)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:790)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 1219224 T1481 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 1219224 T1481 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.highlight.HighlighterTest-918F00AE3D901920-001/init-core-data-001/index [CachedDir<<refCount=0;path=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.highlight.HighlighterTest-918F00AE3D901920-001/init-core-data-001/index;done=false>>]
   [junit4]   2> 1219224 T1481 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.highlight.HighlighterTest-918F00AE3D901920-001/init-core-data-001/index
   [junit4]   2> 1219225 T1481 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.highlight.HighlighterTest-918F00AE3D901920-001/init-core-data-001 [CachedDir<<refCount=0;path=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.highlight.HighlighterTest-918F00AE3D901920-001/init-core-data-001;done=false>>]
   [junit4]   2> 1219225 T1481 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.highlight.HighlighterTest-918F00AE3D901920-001/init-core-data-001
   [junit4]   2> NOTE: leaving temporary files on disk at: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.highlight.HighlighterTest-918F00AE3D901920-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene50): {multiDefault=PostingsFormat(name=Memory doPackFST= false), t_text=PostingsFormat(name=Memory doPackFST= false), foo_s=PostingsFormat(name=LuceneVarGapFixedInterval), subword=PostingsFormat(name=LuceneVarGapFixedInterval), title=PostingsFormat(name=Memory doPackFST= false), foo_t=FSTOrd50, weight=PostingsFormat(name=Memory doPackFST= false), t_text2=PostingsFormat(name=LuceneVarGapFixedInterval), subword_offsets=PostingsFormat(name=MockRandom), id=FSTOrd50, foo_sI=PostingsFormat(name=MockRandom), title_stringNoNorms=PostingsFormat(name=LuceneVarGapFixedInterval), cat=PostingsFormat(name=LuceneVarGapFixedInterval), title_stemmed=PostingsFormat(name=Memory doPackFST= false), title_lettertok=FSTOrd50, intDefault=PostingsFormat(name=LuceneVarGapFixedInterval), tv_text=PostingsFormat(name=LuceneVarGapFixedInterval), _version_=PostingsFormat(name=LuceneVarGapFixedInterval), t_text1=FSTOrd50, textgap=FSTOrd50, text=PostingsFormat(name=MockRandom), tv_no_off_text=PostingsFormat(name=LuceneVarGapFixedInterval), range_facet_l=PostingsFormat(name=LuceneVarGapFixedInterval), timestamp=PostingsFormat(name=LuceneVarGapFixedInterval), tv_mv_text=PostingsFormat(name=LuceneVarGapFixedInterval)}, docValues:{timestamp=DocValuesFormat(name=Direct)}, sim=DefaultSimilarity, locale=de, timezone=Mideast/Riyadh87
   [junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_65 (64-bit)/cpus=16,threads=1,free=188116792,total=247463936
   [junit4]   2> NOTE: All tests run in this JVM: [OverriddenZkACLAndCredentialsProvidersTest, TestConfigSets, SpellCheckCollatorTest, BlockCacheTest, ZkNodePropsTest, TestReplicationHandler, CopyFieldTest, TestFieldSortValues, DateMathParserTest, TestJmxMonitoredMap, TestAnalyzeInfixSuggestions, TestDistribDocBasedVersion, ExternalCollectionsTest, TestReversedWildcardFilterFactory, HighlighterTest]
   [junit4] Completed on J2 in 3.94s, 31 tests, 9 errors <<< FAILURES!

[...truncated 1022 lines...]
BUILD FAILED
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:532: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:473: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:61: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/extra-targets.xml:39: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:508: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1363: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:970: There were test failures: 450 suites, 1841 tests, 9 errors, 56 ignored (10 assumptions)

Total time: 168 minutes 39 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #680
Archived 5 artifacts
Archive block size is 32768
Received 0 blocks and 930484973 bytes
Compression is 0.0%
Took 2 min 45 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Re: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 686 - Still Failing

Posted by Robert Muir <rc...@gmail.com>.
I committed a fix.

On Thu, Nov 27, 2014 at 4:34 PM, Robert Muir <rc...@gmail.com> wrote:
> I inserted DirectDocValues into the randomization... so we might find
> some new failures. I will look into it.
>
> On Thu, Nov 27, 2014 at 4:10 PM, Apache Jenkins Server
> <je...@builds.apache.org> wrote:
>> Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/686/
>>
>> 1 tests failed.
>> REGRESSION:  org.apache.lucene.uninverting.TestFieldCacheVsDocValues.testHugeBinaryValues
>>
>> Error Message:
>>
>>
>> Stack Trace:
>> java.lang.AssertionError
>>         at __randomizedtesting.SeedInfo.seed([BF7BD4B8CA730C54:5C4007BAD241DCDB]:0)
>>         at org.junit.Assert.fail(Assert.java:92)
>>         at org.junit.Assert.assertTrue(Assert.java:43)
>>         at org.junit.Assert.assertTrue(Assert.java:54)
>>         at org.apache.lucene.uninverting.TestFieldCacheVsDocValues.testHugeBinaryValues(TestFieldCacheVsDocValues.java:204)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>         at java.lang.reflect.Method.invoke(Method.java:606)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
>>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>         at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
>>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>>         at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>>         at java.lang.Thread.run(Thread.java:745)
>>
>>
>>
>>
>> Build Log:
>> [...truncated 7576 lines...]
>>    [junit4] Suite: org.apache.lucene.uninverting.TestFieldCacheVsDocValues
>>    [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
>>    [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestFieldCacheVsDocValues -Dtests.method=testHugeBinaryValues -Dtests.seed=BF7BD4B8CA730C54 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=ar_BH -Dtests.timezone=Asia/Calcutta -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>>    [junit4] FAILURE 0.24s J3 | TestFieldCacheVsDocValues.testHugeBinaryValues <<<
>>    [junit4]    > Throwable #1: java.lang.AssertionError
>>    [junit4]    >        at __randomizedtesting.SeedInfo.seed([BF7BD4B8CA730C54:5C4007BAD241DCDB]:0)
>>    [junit4]    >        at org.apache.lucene.uninverting.TestFieldCacheVsDocValues.testHugeBinaryValues(TestFieldCacheVsDocValues.java:204)
>>    [junit4]    >        at java.lang.Thread.run(Thread.java:745)
>>    [junit4]   2> NOTE: leaving temporary files on disk at: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/misc/test/J3/temp/lucene.uninverting.TestFieldCacheVsDocValues-BF7BD4B8CA730C54-001
>>    [junit4]   2> NOTE: test params are: codec=Asserting(Lucene50): {indexed=PostingsFormat(name=Asserting), id=PostingsFormat(name=Asserting)}, docValues:{dv=DocValuesFormat(name=Direct), field=DocValuesFormat(name=Direct)}, sim=DefaultSimilarity, locale=ar_BH, timezone=Asia/Calcutta
>>    [junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_65 (64-bit)/cpus=16,threads=1,free=88834864,total=180355072
>>    [junit4]   2> NOTE: All tests run in this JVM: [TestUninvertingReader, TestSortingMergePolicy, TestDocTermOrds, TestFieldCache, TestFieldCacheSort, TestFieldCacheVsDocValues]
>>    [junit4] Completed on J3 in 64.38s, 10 tests, 1 failure <<< FAILURES!
>>
>> [...truncated 16 lines...]
>> BUILD FAILED
>> /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:532: The following error occurred while executing this line:
>> /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:473: The following error occurred while executing this line:
>> /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:61: The following error occurred while executing this line:
>> /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/extra-targets.xml:39: The following error occurred while executing this line:
>> /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:452: The following error occurred while executing this line:
>> /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2145: The following error occurred while executing this line:
>> /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
>> /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1363: The following error occurred while executing this line:
>> /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:970: There were test failures: 24 suites, 168 tests, 1 failure
>>
>> Total time: 107 minutes 7 seconds
>> Build step 'Invoke Ant' marked build as failure
>> Archiving artifacts
>> Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #680
>> Archived 5 artifacts
>> Archive block size is 32768
>> Received 0 blocks and 223141 bytes
>> Compression is 0.0%
>> Took 0.24 sec
>> Recording test results
>> Email was triggered for: Failure
>> Sending email for trigger: Failure
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Re: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 686 - Still Failing

Posted by Robert Muir <rc...@gmail.com>.
I inserted DirectDocValues into the randomization... so we might find
some new failures. I will look into it.

On Thu, Nov 27, 2014 at 4:10 PM, Apache Jenkins Server
<je...@builds.apache.org> wrote:
> Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/686/
>
> 1 tests failed.
> REGRESSION:  org.apache.lucene.uninverting.TestFieldCacheVsDocValues.testHugeBinaryValues
>
> Error Message:
>
>
> Stack Trace:
> java.lang.AssertionError
>         at __randomizedtesting.SeedInfo.seed([BF7BD4B8CA730C54:5C4007BAD241DCDB]:0)
>         at org.junit.Assert.fail(Assert.java:92)
>         at org.junit.Assert.assertTrue(Assert.java:43)
>         at org.junit.Assert.assertTrue(Assert.java:54)
>         at org.apache.lucene.uninverting.TestFieldCacheVsDocValues.testHugeBinaryValues(TestFieldCacheVsDocValues.java:204)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>         at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>         at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>         at java.lang.Thread.run(Thread.java:745)
>
>
>
>
> Build Log:
> [...truncated 7576 lines...]
>    [junit4] Suite: org.apache.lucene.uninverting.TestFieldCacheVsDocValues
>    [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
>    [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestFieldCacheVsDocValues -Dtests.method=testHugeBinaryValues -Dtests.seed=BF7BD4B8CA730C54 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=ar_BH -Dtests.timezone=Asia/Calcutta -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>    [junit4] FAILURE 0.24s J3 | TestFieldCacheVsDocValues.testHugeBinaryValues <<<
>    [junit4]    > Throwable #1: java.lang.AssertionError
>    [junit4]    >        at __randomizedtesting.SeedInfo.seed([BF7BD4B8CA730C54:5C4007BAD241DCDB]:0)
>    [junit4]    >        at org.apache.lucene.uninverting.TestFieldCacheVsDocValues.testHugeBinaryValues(TestFieldCacheVsDocValues.java:204)
>    [junit4]    >        at java.lang.Thread.run(Thread.java:745)
>    [junit4]   2> NOTE: leaving temporary files on disk at: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/misc/test/J3/temp/lucene.uninverting.TestFieldCacheVsDocValues-BF7BD4B8CA730C54-001
>    [junit4]   2> NOTE: test params are: codec=Asserting(Lucene50): {indexed=PostingsFormat(name=Asserting), id=PostingsFormat(name=Asserting)}, docValues:{dv=DocValuesFormat(name=Direct), field=DocValuesFormat(name=Direct)}, sim=DefaultSimilarity, locale=ar_BH, timezone=Asia/Calcutta
>    [junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_65 (64-bit)/cpus=16,threads=1,free=88834864,total=180355072
>    [junit4]   2> NOTE: All tests run in this JVM: [TestUninvertingReader, TestSortingMergePolicy, TestDocTermOrds, TestFieldCache, TestFieldCacheSort, TestFieldCacheVsDocValues]
>    [junit4] Completed on J3 in 64.38s, 10 tests, 1 failure <<< FAILURES!
>
> [...truncated 16 lines...]
> BUILD FAILED
> /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:532: The following error occurred while executing this line:
> /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:473: The following error occurred while executing this line:
> /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:61: The following error occurred while executing this line:
> /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/extra-targets.xml:39: The following error occurred while executing this line:
> /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:452: The following error occurred while executing this line:
> /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2145: The following error occurred while executing this line:
> /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
> /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1363: The following error occurred while executing this line:
> /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:970: There were test failures: 24 suites, 168 tests, 1 failure
>
> Total time: 107 minutes 7 seconds
> Build step 'Invoke Ant' marked build as failure
> Archiving artifacts
> Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #680
> Archived 5 artifacts
> Archive block size is 32768
> Received 0 blocks and 223141 bytes
> Compression is 0.0%
> Took 0.24 sec
> Recording test results
> Email was triggered for: Failure
> Sending email for trigger: Failure
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 686 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/686/

1 tests failed.
REGRESSION:  org.apache.lucene.uninverting.TestFieldCacheVsDocValues.testHugeBinaryValues

Error Message:


Stack Trace:
java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([BF7BD4B8CA730C54:5C4007BAD241DCDB]:0)
	at org.junit.Assert.fail(Assert.java:92)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertTrue(Assert.java:54)
	at org.apache.lucene.uninverting.TestFieldCacheVsDocValues.testHugeBinaryValues(TestFieldCacheVsDocValues.java:204)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 7576 lines...]
   [junit4] Suite: org.apache.lucene.uninverting.TestFieldCacheVsDocValues
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestFieldCacheVsDocValues -Dtests.method=testHugeBinaryValues -Dtests.seed=BF7BD4B8CA730C54 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=ar_BH -Dtests.timezone=Asia/Calcutta -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 0.24s J3 | TestFieldCacheVsDocValues.testHugeBinaryValues <<<
   [junit4]    > Throwable #1: java.lang.AssertionError
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([BF7BD4B8CA730C54:5C4007BAD241DCDB]:0)
   [junit4]    > 	at org.apache.lucene.uninverting.TestFieldCacheVsDocValues.testHugeBinaryValues(TestFieldCacheVsDocValues.java:204)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: leaving temporary files on disk at: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/misc/test/J3/temp/lucene.uninverting.TestFieldCacheVsDocValues-BF7BD4B8CA730C54-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene50): {indexed=PostingsFormat(name=Asserting), id=PostingsFormat(name=Asserting)}, docValues:{dv=DocValuesFormat(name=Direct), field=DocValuesFormat(name=Direct)}, sim=DefaultSimilarity, locale=ar_BH, timezone=Asia/Calcutta
   [junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_65 (64-bit)/cpus=16,threads=1,free=88834864,total=180355072
   [junit4]   2> NOTE: All tests run in this JVM: [TestUninvertingReader, TestSortingMergePolicy, TestDocTermOrds, TestFieldCache, TestFieldCacheSort, TestFieldCacheVsDocValues]
   [junit4] Completed on J3 in 64.38s, 10 tests, 1 failure <<< FAILURES!

[...truncated 16 lines...]
BUILD FAILED
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:532: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:473: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:61: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/extra-targets.xml:39: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:452: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2145: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1363: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:970: There were test failures: 24 suites, 168 tests, 1 failure

Total time: 107 minutes 7 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #680
Archived 5 artifacts
Archive block size is 32768
Received 0 blocks and 223141 bytes
Compression is 0.0%
Took 0.24 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 685 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/685/

No tests ran.

Build Log:
[...truncated 260 lines...]
    [javac] Compiling 146 source files to /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/test-framework/classes/java
    [javac] /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/test-framework/src/java/org/apache/lucene/mockfile/FilterFileSystem.java:104: error: <anonymous org.apache.lucene.mockfile.FilterFileSystem$1$1> is not abstract and does not override abstract method remove() in Iterator
    [javac]         return new Iterator<Path>() {
    [javac]                                     ^
    [javac] /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/test-framework/src/java/org/apache/lucene/mockfile/FilterFileSystem.java:126: error: <anonymous org.apache.lucene.mockfile.FilterFileSystem$2$1> is not abstract and does not override abstract method remove() in Iterator
    [javac]         return new Iterator<FileStore>() {
    [javac]                                          ^
    [javac] /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/test-framework/src/java/org/apache/lucene/mockfile/FilterPath.java:230: error: <anonymous org.apache.lucene.mockfile.FilterPath$1> is not abstract and does not override abstract method remove() in Iterator
    [javac]     return new Iterator<Path>() {
    [javac]                                 ^
    [javac] Note: Some input files use or override a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
    [javac] 3 errors

BUILD FAILED
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:532: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:473: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:61: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/extra-targets.xml:39: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:49: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:765: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:514: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1870: Compile failed; see the compiler error output for details.

Total time: 14 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #680
Archived 1 artifacts
Archive block size is 32768
Received 0 blocks and 464 bytes
Compression is 0.0%
Took 75 ms
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 684 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/684/

1 tests failed.
REGRESSION:  org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testDistribSearch

Error Message:
Error executing query

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Error executing query
	at __randomizedtesting.SeedInfo.seed([4B0EDA6A9A7D3207:CAE85472ED22523B]:0)
	at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:100)
	at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:301)
	at org.apache.solr.cloud.CloudInspectUtil.compareResults(CloudInspectUtil.java:223)
	at org.apache.solr.cloud.CloudInspectUtil.compareResults(CloudInspectUtil.java:165)
	at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testIndexingBatchPerRequestWithHttpSolrServer(FullSolrCloudDistribCmdsTest.java:414)
	at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.doTest(FullSolrCloudDistribCmdsTest.java:144)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.http.ParseException: Invalid content type: 
	at org.apache.http.entity.ContentType.parse(ContentType.java:233)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:513)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:215)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:211)
	at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:91)
	... 46 more




Build Log:
[...truncated 12202 lines...]
   [junit4] Suite: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
   [junit4]   2> Creating dataDir: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/init-core-data-001
   [junit4]   2> 2060801 T4963 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
   [junit4]   2> 2060805 T4963 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> 2060806 T4963 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2060807 T4964 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 2060907 T4963 oasc.ZkTestServer.run start zk server on port:16643
   [junit4]   2> 2060908 T4963 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2060909 T4963 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2060913 T4971 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1760cbe4 name:ZooKeeperConnection Watcher:127.0.0.1:16643 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2060913 T4963 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2060914 T4963 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2060914 T4963 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 2060916 T4963 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2060917 T4963 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2060919 T4974 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6def37c name:ZooKeeperConnection Watcher:127.0.0.1:16643/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2060920 T4963 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2060920 T4963 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2060920 T4963 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 2060922 T4963 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 2060924 T4963 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 2060925 T4963 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 2060927 T4963 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 2060928 T4963 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 2060930 T4963 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 2060931 T4963 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 2061037 T4963 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2061038 T4963 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2061040 T4963 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 2061041 T4963 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 2061043 T4963 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 2061044 T4963 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 2061046 T4963 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 2061046 T4963 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 2061049 T4963 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 2061049 T4963 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 2061051 T4963 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2061052 T4963 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2061054 T4963 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2061055 T4963 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2061057 T4963 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 2061057 T4963 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 2061059 T4963 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 2061060 T4963 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 2062623 T4963 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 2062628 T4963 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:60854
   [junit4]   2> 2062628 T4963 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 2062629 T4963 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2062629 T4963 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-002
   [junit4]   2> 2062630 T4963 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-002/'
   [junit4]   2> 2062679 T4963 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-002/solr.xml
   [junit4]   2> 2062766 T4963 oasc.CoreContainer.<init> New CoreContainer 968261613
   [junit4]   2> 2062767 T4963 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-002/]
   [junit4]   2> 2062768 T4963 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 2062768 T4963 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 2062769 T4963 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 2062769 T4963 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 2062769 T4963 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 2062770 T4963 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 2062770 T4963 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 2062770 T4963 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 2062771 T4963 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 2062771 T4963 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 2062772 T4963 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 2062772 T4963 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2062772 T4963 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2062773 T4963 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 2062773 T4963 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:16643/solr
   [junit4]   2> 2062774 T4963 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 2062774 T4963 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2062775 T4963 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2062777 T4986 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11d0b9dc name:ZooKeeperConnection Watcher:127.0.0.1:16643 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2062778 T4963 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2062778 T4963 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2062780 T4963 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2062782 T4989 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1dae307b name:ZooKeeperConnection Watcher:127.0.0.1:16643/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2062783 T4963 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2062784 T4963 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 2062787 T4963 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 2062789 T4963 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 2062792 T4963 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 2062794 T4963 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 2062796 T4963 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 2062798 T4963 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60854_
   [junit4]   2> 2062799 T4963 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:60854_
   [junit4]   2> 2062801 T4963 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 2062802 T4963 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 2062804 T4963 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 2062806 T4963 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:60854_
   [junit4]   2> 2062806 T4963 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 2062808 T4963 oasc.Overseer.start Overseer (id=92860478844370947-127.0.0.1:60854_-n_0000000000) starting
   [junit4]   2> 2062810 T4963 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 2062817 T4963 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 2062818 T4991 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 2062818 T4963 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 2062820 T4963 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 2062821 T4963 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2062824 T4990 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 2062830 T4993 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
   [junit4]   2> 2062830 T4993 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 2062831 T4989 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 2062831 T4993 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 2062833 T4990 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:60854",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "node_name":"127.0.0.1:60854_",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "shard":null}
   [junit4]   2> 2062834 T4990 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
   [junit4]   2> 2062834 T4990 oasc.Overseer$ClusterStateUpdater.createCollection state version control_collection 1
   [junit4]   2> 2062834 T4990 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 2062836 T4989 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2063831 T4993 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2063832 T4993 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
   [junit4]   2> 2063833 T4993 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 2063833 T4993 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 2063834 T4993 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2063834 T4993 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-002/collection1/'
   [junit4]   2> 2063836 T4993 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-002/collection1/lib/classes/' to classloader
   [junit4]   2> 2063837 T4993 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-002/collection1/lib/README' to classloader
   [junit4]   2> 2063837 T4993 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-002/collection1/lib/.svn/' to classloader
   [junit4]   2> 2063870 T4993 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 2063873 T4993 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 2063906 T4993 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 2063952 T4993 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2064053 T4993 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 2064075 T4993 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 2064393 T4993 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 2064395 T4993 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 2064397 T4993 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2064402 T4993 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2064418 T4993 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 2064418 T4993 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2064419 T4993 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-002/collection1/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-001/control/data/
   [junit4]   2> 2064419 T4993 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7309eabd
   [junit4]   2> 2064420 T4993 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-001/control/data
   [junit4]   2> 2064420 T4993 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-001/control/data/index/
   [junit4]   2> 2064420 T4993 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2064421 T4993 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-001/control/data/index
   [junit4]   2> 2064421 T4993 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=42, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=2.0048828125, floorSegmentMB=0.654296875, forceMergeDeletesPctAllowed=9.507636286823223, segmentsPerTier=27.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 2064422 T4993 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=BaseDirectoryWrapper(RAMDirectory@6bf442f0 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7156d1ee),segFN=segments_1,generation=1}
   [junit4]   2> 2064422 T4993 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2064426 T4993 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2064427 T4993 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2064427 T4993 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2064427 T4993 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2064428 T4993 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2064428 T4993 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2064428 T4993 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2064429 T4993 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2064429 T4993 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2064429 T4993 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler
   [junit4]   2> 2064430 T4993 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 2064430 T4993 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 2064431 T4993 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 2064431 T4993 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 2064432 T4993 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 2064432 T4993 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 2064432 T4993 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 2064433 T4993 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 2064433 T4993 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 2064433 T4993 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 2064434 T4993 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 2064452 T4993 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2064454 T4993 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2064456 T4993 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2064458 T4993 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2064464 T4993 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 2064464 T4993 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 2064464 T4993 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=12, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6574484810602126]
   [junit4]   2> 2064465 T4993 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=BaseDirectoryWrapper(RAMDirectory@6bf442f0 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7156d1ee),segFN=segments_1,generation=1}
   [junit4]   2> 2064465 T4993 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2064466 T4993 oass.SolrIndexSearcher.<init> Opening Searcher@6fa504bb[collection1] main
   [junit4]   2> 2064466 T4993 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 2064467 T4993 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2064467 T4993 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 2064467 T4993 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2064467 T4993 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 2064468 T4993 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4] HEARTBEAT J3 PID(3814@lucene.zones.apache.org): 2014-11-25T18:03:54, stalled for 60.1s at: OverseerStatusTest.testDistribSearch
   [junit4]   2> 2064468 T4993 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 2064468 T4993 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2064469 T4993 oascc.SolrZkClient.makePath makePath: /configs/conf1/_rest_managed.json
   [junit4]   2> 2064471 T4993 oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new znode /configs/conf1/_rest_managed.json
   [junit4]   2> 2064471 T4993 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2064471 T4993 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 2064474 T4994 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6fa504bb[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2064474 T4993 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 2064475 T4997 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:60854 collection:control_collection shard:shard1
   [junit4]   2> 2064476 T4963 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 2064476 T4997 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 2064476 T4963 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 2064477 T4963 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2064478 T4963 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2064479 T5000 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2fd9f443 name:ZooKeeperConnection Watcher:127.0.0.1:16643/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2064480 T4963 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2064480 T4963 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2064481 T4997 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 2064481 T4963 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2064482 T4989 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 2064482 T4997 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 2064482 T4997 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C896 name=collection1 org.apache.solr.core.SolrCore@60b05b34 url=http://127.0.0.1:60854/collection1 node=127.0.0.1:60854_ C896_STATE=coll:control_collection core:collection1 props:{state=down, core=collection1, base_url=http://127.0.0.1:60854, node_name=127.0.0.1:60854_}
   [junit4]   2> 2064483 T4997 C896 P60854 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:60854/collection1/
   [junit4]   2> 2064483 T4997 C896 P60854 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 2064483 T4997 C896 P60854 oasc.SyncStrategy.syncToMe http://127.0.0.1:60854/collection1/ has no replicas
   [junit4]   2> 2064483 T4997 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:60854/collection1/ shard1
   [junit4]   2> 2064483 T4963 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 2064484 T4997 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 2064484 T4963 oasc.AbstractFullDistribZkTestBase.createJettys Creating collection1 with stateFormat=2
   [junit4]   2> 2064484 T4990 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection"}
   [junit4]   2> 2064484 T4963 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2064485 T4963 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2064486 T5003 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4c4a75e1 name:ZooKeeperConnection Watcher:127.0.0.1:16643/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2064487 T4963 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2064487 T4963 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2064487 T5000 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2064487 T4989 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2064491 T4990 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 2, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:60854",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"active"}
   [junit4]   2> 2064493 T4989 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 2064493 T4990 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 2, message = {
   [junit4]   2> 	  "operation":"create",
   [junit4]   2> 	  "name":"collection1",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "stateFormat":2}
   [junit4]   2> 2064493 T4990 oasc.Overseer$ClusterStateUpdater.buildCollection Building a new collection: collection1
   [junit4]   2> 2064493 T4990 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3]
   [junit4]   2> 2064494 T4990 oasc.Overseer$ClusterStateUpdater.createCollection state version collection1 2
   [junit4]   2> 2064495 T4990 oasc.Overseer$ClusterStateUpdater.updateZkStates going to create_collection /collections/collection1/state.json
   [junit4]   2> 2064496 T5000 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2064497 T4989 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2064539 T4997 oasc.ZkController.register We are http://127.0.0.1:60854/collection1/ and leader is http://127.0.0.1:60854/collection1/
   [junit4]   2> 2064540 T4997 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:60854
   [junit4]   2> 2064540 T4997 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 2064540 T4997 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
   [junit4]   2> 2064540 T4997 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 2064541 T4989 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 2064544 T4990 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:60854",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "node_name":"127.0.0.1:60854_",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "shard":"shard1"}
   [junit4]   2> 2064647 T4989 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2064647 T5000 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2065554 T4963 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
   [junit4]   2> 2065555 T4963 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 2065557 T4963 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:41803
   [junit4]   2> 2065558 T4963 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 2065558 T4963 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2065558 T4963 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-003
   [junit4]   2> 2065558 T4963 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-003/'
   [junit4]   2> 2065602 T4963 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-003/solr.xml
   [junit4]   2> 2065655 T4963 oasc.CoreContainer.<init> New CoreContainer 2119065287
   [junit4]   2> 2065655 T4963 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-003/]
   [junit4]   2> 2065656 T4963 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 2065656 T4963 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 2065656 T4963 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 2065657 T4963 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 2065657 T4963 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 2065657 T4963 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 2065657 T4963 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 2065658 T4963 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 2065658 T4963 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 2065658 T4963 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 2065658 T4963 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 2065659 T4963 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2065659 T4963 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2065659 T4963 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 2065659 T4963 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:16643/solr
   [junit4]   2> 2065660 T4963 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 2065660 T4963 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2065661 T4963 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2065663 T5015 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7358ba7b name:ZooKeeperConnection Watcher:127.0.0.1:16643 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2065664 T4963 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2065664 T4963 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2065665 T4963 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2065667 T5018 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5da87351 name:ZooKeeperConnection Watcher:127.0.0.1:16643/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2065667 T4963 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2065671 T4963 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2066676 T4963 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41803_
   [junit4]   2> 2066677 T4963 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:41803_
   [junit4]   2> 2066680 T4963 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 2066688 T5019 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 2066688 T5019 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 2066690 T4989 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 2066690 T5019 oasc.ZkController.preRegister Registering watch for external collection collection1
   [junit4]   2> 2066690 T5019 oascc.ZkStateReader.addZkWatch addZkWatch collection1
   [junit4]   2> 2066691 T5019 oascc.ZkStateReader.addZkWatch Updating collection state at /collections/collection1/state.json from ZooKeeper... 
   [junit4]   2> 2066692 T5019 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 0 
   [junit4]   2> 2066692 T5019 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 2066694 T4990 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:41803",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "node_name":"127.0.0.1:41803_",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "shard":null}
   [junit4]   2> 2066696 T4990 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
   [junit4]   2> 2066697 T4990 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 2066700 T4989 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 2066701 T4990 oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection /collections/collection1/state.json
   [junit4]   2> 2066701 T5018 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json, has occurred - updating... 
   [junit4]   2> 2066702 T5018 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 1 
   [junit4]   2> 2067692 T5019 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2067693 T5019 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 2067693 T5019 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 2067693 T5019 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 2067694 T5019 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2067694 T5019 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-003/collection1/'
   [junit4]   2> 2067696 T5019 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-003/collection1/lib/.svn/' to classloader
   [junit4]   2> 2067696 T5019 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-003/collection1/lib/README' to classloader
   [junit4]   2> 2067697 T5019 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-003/collection1/lib/classes/' to classloader
   [junit4]   2> 2067721 T5019 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 2067723 T5019 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 2067747 T5019 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 2067779 T5019 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2067880 T5019 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 2067886 T5019 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 2068120 T5019 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 2068122 T5019 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 2068123 T5019 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2068126 T5019 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2068139 T5019 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 2068140 T5019 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2068140 T5019 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-003/collection1/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-001/jetty1/
   [junit4]   2> 2068140 T5019 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7309eabd
   [junit4]   2> 2068141 T5019 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-001/jetty1
   [junit4]   2> 2068141 T5019 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-001/jetty1/index/
   [junit4]   2> 2068141 T5019 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-001/jetty1/index' doesn't exist. Creating new index...
   [junit4]   2> 2068142 T5019 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-001/jetty1/index
   [junit4]   2> 2068142 T5019 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=42, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=2.0048828125, floorSegmentMB=0.654296875, forceMergeDeletesPctAllowed=9.507636286823223, segmentsPerTier=27.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 2068142 T5019 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=BaseDirectoryWrapper(RAMDirectory@651c187b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@178b6719),segFN=segments_1,generation=1}
   [junit4]   2> 2068143 T5019 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2068146 T5019 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2068147 T5019 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2068147 T5019 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2068147 T5019 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2068147 T5019 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2068147 T5019 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2068148 T5019 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2068148 T5019 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2068148 T5019 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2068149 T5019 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler
   [junit4]   2> 2068149 T5019 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 2068149 T5019 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 2068150 T5019 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 2068150 T5019 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 2068150 T5019 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 2068151 T5019 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 2068151 T5019 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 2068151 T5019 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 2068151 T5019 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 2068152 T5019 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 2068152 T5019 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 2068166 T5019 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2068167 T5019 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2068169 T5019 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2068170 T5019 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2068173 T5019 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 2068173 T5019 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 2068174 T5019 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=12, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6574484810602126]
   [junit4]   2> 2068174 T5019 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=BaseDirectoryWrapper(RAMDirectory@651c187b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@178b6719),segFN=segments_1,generation=1}
   [junit4]   2> 2068174 T5019 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2068175 T5019 oass.SolrIndexSearcher.<init> Opening Searcher@6da2f2ba[collection1] main
   [junit4]   2> 2068175 T5019 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 2068176 T5019 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2068176 T5019 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 2068176 T5019 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2068176 T5019 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 2068176 T5019 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2068177 T5019 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
   [junit4]   2> 2068177 T5019 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2068177 T5019 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
   [junit4]   2> 2068177 T5019 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 2068181 T5020 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6da2f2ba[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2068181 T5019 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 2068182 T5023 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:41803 collection:collection1 shard:shard2
   [junit4]   2> 2068182 T4963 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 2068182 T4963 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 2068182 T5023 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
   [junit4]   2> 2068186 T5023 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
   [junit4]   2> 2068188 T4989 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 2068188 T5023 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 2068188 T5023 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C897 name=collection1 org.apache.solr.core.SolrCore@5a23d657 url=http://127.0.0.1:41803/collection1 node=127.0.0.1:41803_ C897_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, base_url=http://127.0.0.1:41803, node_name=127.0.0.1:41803_}
   [junit4]   2> 2068188 T5023 C897 P41803 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:41803/collection1/
   [junit4]   2> 2068189 T5023 C897 P41803 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 2068189 T5023 C897 P41803 oasc.SyncStrategy.syncToMe http://127.0.0.1:41803/collection1/ has no replicas
   [junit4]   2> 2068189 T5023 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:41803/collection1/ shard2
   [junit4]   2> 2068189 T5023 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
   [junit4]   2> 2068190 T4990 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"collection1"}
   [junit4]   2> 2068193 T4990 oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection /collections/collection1/state.json
   [junit4]   2> 2068193 T5018 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json, has occurred - updating... 
   [junit4]   2> 2068194 T5018 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 2 
   [junit4]   2> 2068197 T4990 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:41803",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"active"}
   [junit4]   2> 2068199 T4989 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 2068199 T4990 oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection /collections/collection1/state.json
   [junit4]   2> 2068199 T5018 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json, has occurred - updating... 
   [junit4]   2> 2068200 T5018 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 3 
   [junit4]   2> 2068243 T5023 oasc.ZkController.register We are http://127.0.0.1:41803/collection1/ and leader is http://127.0.0.1:41803/collection1/
   [junit4]   2> 2068244 T5023 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:41803
   [junit4]   2> 2068244 T5023 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 2068244 T5023 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
   [junit4]   2> 2068244 T5023 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 2068245 T4989 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 2068247 T5023 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 3 
   [junit4]   2> 2068248 T4990 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:41803",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "node_name":"127.0.0.1:41803_",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "shard":"shard2"}
   [junit4]   2> 2068251 T4990 oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection /collections/collection1/state.json
   [junit4]   2> 2068252 T5018 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json, has occurred - updating... 
   [junit4]   2> 2068252 T5018 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 4 
   [junit4]   2> 2069839 T4963 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
   [junit4]   2> 2069840 T4963 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 2069843 T4963 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:54094
   [junit4]   2> 2069844 T4963 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 2069844 T4963 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2069845 T4963 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-004
   [junit4]   2> 2069845 T4963 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-004/'
   [junit4]   2> 2069910 T4963 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-004/solr.xml
   [junit4]   2> 2070036 T4963 oasc.CoreContainer.<init> New CoreContainer 1515531269
   [junit4]   2> 2070037 T4963 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-004/]
   [junit4]   2> 2070038 T4963 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 2070039 T4963 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 2070039 T4963 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 2070039 T4963 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 2070040 T4963 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 2070040 T4963 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 2070040 T4963 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 2070041 T4963 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 2070041 T4963 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 2070041 T4963 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 2070042 T4963 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 2070042 T4963 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2070043 T4963 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2070043 T4963 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 2070043 T4963 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:16643/solr
   [junit4]   2> 2070044 T4963 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 2070044 T4963 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2070045 T4963 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2070047 T5035 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14c7a826 name:ZooKeeperConnection Watcher:127.0.0.1:16643 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2070048 T4963 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2070048 T4963 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2070050 T4963 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2070052 T5038 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7ae6da87 name:ZooKeeperConnection Watcher:127.0.0.1:16643/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2070052 T4963 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2070057 T4963 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2071062 T4963 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54094_
   [junit4]   2> 2071063 T4963 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:54094_
   [junit4]   2> 2071066 T4963 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 2071074 T5039 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 2071074 T5039 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 2071076 T4989 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 2071077 T5039 oasc.ZkController.preRegister Registering watch for external collection collection1
   [junit4]   2> 2071077 T5039 oascc.ZkStateReader.addZkWatch addZkWatch collection1
   [junit4]   2> 2071077 T5039 oascc.ZkStateReader.addZkWatch Updating collection state at /collections/collection1/state.json from ZooKeeper... 
   [junit4]   2> 2071078 T5039 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 4 
   [junit4]   2> 2071078 T4990 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:54094",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "node_name":"127.0.0.1:54094_",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "shard":null}
   [junit4]   2> 2071079 T5039 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 2071081 T4990 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
   [junit4]   2> 2071081 T4990 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
   [junit4]   2> 2071085 T4990 oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection /collections/collection1/state.json
   [junit4]   2> 2071085 T5018 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json, has occurred - updating... 
   [junit4]   2> 2071085 T5038 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json, has occurred - updating... 
   [junit4]   2> 2071086 T5018 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 5 
   [junit4]   2> 2071086 T5038 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 5 
   [junit4]   2> 2072079 T5039 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2072080 T5039 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 2072081 T5039 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 2072081 T5039 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 2072082 T5039 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2072082 T5039 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-004/collection1/'
   [junit4]   2> 2072083 T5039 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-004/collection1/lib/README' to classloader
   [junit4]   2> 2072084 T5039 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-004/collection1/lib/classes/' to classloader
   [junit4]   2> 2072085 T5039 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-004/collection1/lib/.svn/' to classloader
   [junit4]   2> 2072118 T5039 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 2072121 T5039 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 2072155 T5039 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 2072201 T5039 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2072302 T5039 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 2072312 T5039 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 2072674 T5039 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 2072677 T5039 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 2072679 T5039 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2072684 T5039 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2072703 T5039 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 2072703 T5039 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2072704 T5039 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-004/collection1/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-001/jetty2/
   [junit4]   2> 2072704 T5039 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7309eabd
   [junit4]   2> 2072705 T5039 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-001/jetty2
   [junit4]   2> 2072705 T5039 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-001/jetty2/index/
   [junit4]   2> 2072705 T5039 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-001/jetty2/index' doesn't exist. Creating new index...
   [junit4]   2> 2072707 T5039 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001/tempDir-001/jetty2/index
   [junit4]   2> 2072707 T5039 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=42, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=2.0048828125, floorSegmentMB=0.654296875, forceMergeDeletesPctAllowed=9.507636286823223, segmentsPerTier=27.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 2072708 T5039 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=BaseDirectoryWrapper(RAMDirectory@72983c4d lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@689f1e01),segFN=segments_1,generation=1}
   [junit4]   2> 2072708 T5039 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2072714 T5039 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2072714 T5039 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2072714 T5039 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2072715 T5039 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2072715 T5039 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2072715 T5039 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2072716 T5039 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2072716 T5039 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2072716 T5039 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2072717 T5039 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler
   [junit4]   2> 2072718 T5039 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 2072718 T5039 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 2072719 T5039 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 2072719 T5039 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 2072720 T5039 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 2072720 T5039 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 2072720 T5039 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 2072721 T5039 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 2072721 T5039 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 2072722 T5039 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 2072722 T5039 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 2072742 T5039 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2072745 T5039 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2072747 T5039 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2072749 T5039 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2072753 T5039 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 2072753 T5039 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 2072755 T5039 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=12, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6574484810602126]
   [junit4]   2> 2072755 T5039 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=BaseDirectoryWrapper(RAMDirectory@72983c4d lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@689f1e01),segFN=segments_1,generation=1}
   [junit4]   2> 2072756 T5039 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2072756 T5039 oass.SolrIndexSearcher.<init> Opening Searcher@1c86aed1[collection1] main
   [junit4]   2> 2072756 T5039 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 2072757 T5039 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2072758 T5039 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 2072758 T5039 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2072758 T5039 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 2072759 T5039 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2072759 T5039 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
   [junit4]   2> 2072760 T5039 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2072760 T5039 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
   [junit4]   2> 2072760 T5039 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 2072764 T5040 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1c86aed1[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2072765 T5039 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 2072765 T5043 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:54094 collection:collection1 shard:shard3
   [junit4]   2> 2072766 T4963 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 2072766 T4963 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 2072766 T5043 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
   [junit4]   2> 2072771 T5043 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard3
   [junit4]   2> 2072773 T4989 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 2072773 T5043 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 2072773 T5043 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C898 name=collection1 org.apache.solr.core.SolrCore@6bed89dc url=http://127.0.0.1:54094/collection1 node=127.0.0.1:54094_ C898_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, base_url=http://127.0.0.1:54094, node_name=127.0.0.1:54094_}
   [junit4]   2> 2072773 T5043 C898 P54094 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:54094/collection1/
   [junit4]   2> 2072774 T5043 C898 P54094 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 2072774 T5043 C898 P54094 oasc.SyncStrategy.syncToMe http://127.0.0.1:54094/collection1/ has no replicas
   [junit4]   2> 2072774 T5043 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:54094/collection1/ shard3
   [junit4]   2> 2072774 T5043 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3
   [junit4]   2> 2072776 T4990 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard3",
   [junit4]   2> 	  "collection":"collection1"}
   [junit4]   2> 2072779 T4990 oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection /collections/collection1/state.json
   [junit4]   2> 2072779 T5038 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json, has occurred - updating... 
   [junit4]   2> 2072779 T5018 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json, has occurred - updating... 
   [junit4]   2> 2072780 T5038 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 6 
   [junit4]   2> 2072780 T5018 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 6 
   [junit4]   2> 2072784 T4990 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard3",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:54094",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"active"}
   [junit4]   2> 2072786 T4989 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 2072787 T4990 oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection /collections/collection1/state.json
   [junit4]   2> 2072787 T5018 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json, has occurred - updating... 
   [junit4]   2> 2072787 T5038 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json, has occurred - updating... 
   [junit4]   2> 2072788 T5018 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 7 
   [junit4]   2> 2072788 T5038 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 7 
   [junit4]   2> 2072829 T5043 oasc.ZkController.register We are http://127.0.0.1:54094/collection1/ and leader is http://127.0.0.1:54094/collection1/
   [junit4]   2> 2072830 T5043 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:54094
   [junit4]   2> 2072830 T5043 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 2072830 T5043 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
   [junit4]   2> 2072830 T5043 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 2072832 T4989 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 2072835 T5043 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 7 
   [junit4]   2> 2072835 T4990 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:54094",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node2",
   [junit4]   2> 	  "node_name":"127.0.0.1:54094_",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "shard":"shard3"}
   [junit4]   2> 2072840 T4990 oasc.Overse

[...truncated too long message...]

FullSolrCloudDistribCmdsTest-4B0EDA6A9A7D3207-001
   [junit4]   2> 174595 T4962 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene50): {17parent_f1_s=Lucene50(blocksize=128), 38parent_f1_s=PostingsFormat(name=MockRandom), 34parent_f2_s=Lucene50(blocksize=128), 4parent_f2_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 22parent_f1_s=Lucene50(blocksize=128), 1parent_f2_s=FST50, 7parent_f2_s=Lucene50(blocksize=128), 29parent_f2_s=Lucene50(blocksize=128), 3child0_s=PostingsFormat(name=MockRandom), type_s=Lucene50(blocksize=128), 9parent_f2_s=FST50, 27parent_f1_s=PostingsFormat(name=MockRandom), 9parent_f1_s=PostingsFormat(name=MockRandom), 2child0_s=FST50, 7grand1_s=FST50, 29parent_f1_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 7grand3_s=Lucene50(blocksize=128), 2parent_f2_s=PostingsFormat(name=MockRandom), 7parent_f1_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 18parent_f2_s=Lucene50(blocksize=128), _root_=Lucene50(blocksize=128), 17parent_f2_s=PostingsFormat(name=MockRandom), 12parent_f1_s=PostingsFormat(name=MockRandom), 3child1_s=FST50, 6parent_f1_s=Lucene50(blocksize=128), 20parent_f2_s=PostingsFormat(name=MockRandom), 38parent_f2_s=Lucene50(blocksize=128), 7grand0_s=PostingsFormat(name=MockRandom), 16parent_f1_s=PostingsFormat(name=MockRandom), 4grand0_s=Lucene50(blocksize=128), 6grand2_s=Lucene50(blocksize=128), 4child0_s=FST50, 6grand1_s=PostingsFormat(name=MockRandom), 31parent_f1_s=FST50, 35parent_f1_s=FST50, 20parent_f1_s=FST50, 25parent_f1_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 14parent_f1_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 37parent_f2_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 25parent_f2_s=FST50, 1parent_f1_s=PostingsFormat(name=MockRandom), 8parent_f1_s=FST50, 21parent_f1_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 1child0_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 7grand6_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 2grand0_s=Lucene50(blocksize=128), 1grand0_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), text_t=Lucene50(blocksize=128), 36parent_f2_s=FST50, 32parent_f1_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 33parent_f1_s=Lucene50(blocksize=128), 39parent_f1_s=FST50, 5parent_f1_s=PostingsFormat(name=MockRandom), 3child2_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 34parent_f1_s=PostingsFormat(name=MockRandom), 5grand3_s=FST50, 13parent_f2_s=PostingsFormat(name=MockRandom), 19parent_f2_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 6grand0_s=FST50, 19parent_f1_s=FST50, 4grand2_s=FST50, 8parent_f2_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 0parent_f2_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), _version_=Lucene50(blocksize=128), 35parent_f2_s=PostingsFormat(name=MockRandom), 24parent_f1_s=FST50, 14parent_f2_s=FST50, 4child1_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 12parent_f2_s=Lucene50(blocksize=128), 15parent_f2_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 3grand2_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 30parent_f1_s=PostingsFormat(name=MockRandom), 7grand5_s=FST50, 6grand5_s=PostingsFormat(name=MockRandom), 2grand1_s=PostingsFormat(name=MockRandom), 28parent_f1_s=Lucene50(blocksize=128), 28parent_f2_s=PostingsFormat(name=MockRandom), 4grand1_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 26parent_f1_s=Lucene50(blocksize=128), 10parent_f1_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 37parent_f1_s=Lucene50(blocksize=128), 5grand2_s=PostingsFormat(name=MockRandom), 23parent_f1_s=PostingsFormat(name=MockRandom), 5grand0_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 6grand4_s=FST50, 0parent_f1_s=FST50, 39parent_f2_s=PostingsFormat(name=MockRandom), a_t=FST50, 11parent_f1_s=Lucene50(blocksize=128), 5grand4_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 7grand2_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 2parent_f1_s=Lucene50(blocksize=128), 3grand1_s=FST50, 4parent_f1_s=FST50, 6parent_f2_s=PostingsFormat(name=MockRandom), 33parent_f2_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 26parent_f2_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 31parent_f2_s=PostingsFormat(name=MockRandom), 11parent_f2_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 32parent_f2_s=FST50, 7grand4_s=PostingsFormat(name=MockRandom), rnd_b=PostingsFormat(name=MockRandom), 6grand3_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 4grand3_s=PostingsFormat(name=MockRandom), 2child1_s=PostingsFormat(name=MockRandom), 22parent_f2_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 13parent_f1_s=FST50, 30parent_f2_s=Lucene50(blocksize=128), 21parent_f2_s=FST50, 18parent_f1_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 16parent_f2_s=FST50, 3parent_f2_s=Lucene50(blocksize=128), 27parent_f2_s=Lucene50(blocksize=128), 4child2_s=Lucene50(blocksize=128), 24parent_f2_s=PostingsFormat(name=MockRandom), 23parent_f2_s=Lucene50(blocksize=128), 3parent_f1_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 15parent_f1_s=Lucene50(blocksize=128), 4child3_s=PostingsFormat(name=MockRandom), 3grand0_s=PostingsFormat(name=MockRandom), id=PostingsFormat(name=MockRandom), 10parent_f2_s=FST50, 5grand1_s=Lucene50(blocksize=128), 5parent_f2_s=FST50, 36parent_f1_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128)))}, docValues:{}, sim=DefaultSimilarity, locale=es_CL, timezone=America/Cuiaba
   [junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_65 (64-bit)/cpus=16,threads=1,free=184736200,total=440401920
   [junit4]   2> NOTE: All tests run in this JVM: [TestFastOutputStream, SortByFunctionTest, TestCollectionAPI, TestShortCircuitedRequests, CachingDirectoryFactoryTest, FastVectorHighlighterTest, TestSolrDynamicMBean, TestStressVersions, LukeRequestHandlerTest, NotRequiredUniqueKeyTest, TestCloudManagedSchema, DocValuesMissingTest, TestDistribDocBasedVersion, SampleTest, SimpleCollectionCreateDeleteTest, TestDefaultSearchFieldResource, SuggestComponentTest, TestBM25SimilarityFactory, CoreAdminCreateDiscoverTest, TestDocumentBuilder, TestAtomicUpdateErrorCases, TestBlendedInfixSuggestions, MBeansHandlerTest, TestOverriddenPrefixQueryForCustomFieldType, TestFunctionQuery, TestReplicaProperties, ExternalCollectionsTest, SolrRequestParserTest, PrimitiveFieldTypeTest, TestCloudInspectUtil, DateFieldTest, TestOrdValues, PathHierarchyTokenizerFactoryTest, AutoCommitTest, ChangedSchemaMergeTest, TestPartialUpdateDeduplication, ExternalFileFieldSortTest, TestRecovery, BlockCacheTest, TestDocBasedVersionConstraints, SpatialFilterTest, MigrateRouteKeyTest, SystemInfoHandlerTest, TestConfig, ClusterStateTest, TestCollationField, ZkControllerTest, TestExceedMaxTermLength, PeerSyncTest, TestCustomSort, DirectUpdateHandlerOptimizeTest, TestSchemaVersionResource, DistributedFacetPivotSmallAdvancedTest, HdfsWriteToMultipleCollectionsTest, BasicDistributedZkTest, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, OverseerTest, LeaderElectionIntegrationTest, BasicZkTest, FullSolrCloudDistribCmdsTest]
   [junit4] Completed on J1 in 175.13s, 1 test, 1 error <<< FAILURES!

[...truncated 840 lines...]
BUILD FAILED
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:532: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:473: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:61: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/extra-targets.xml:39: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:508: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1363: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:970: There were test failures: 449 suites, 1825 tests, 1 error, 55 ignored (9 assumptions)

Total time: 208 minutes 3 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #680
Archived 5 artifacts
Archive block size is 32768
Received 0 blocks and 722886541 bytes
Compression is 0.0%
Took 2 min 17 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 683 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/683/

1 tests failed.
REGRESSION:  org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.testDistribSearch

Error Message:
There were too many update fails - we expect it can happen, but shouldn't easily

Stack Trace:
java.lang.AssertionError: There were too many update fails - we expect it can happen, but shouldn't easily
	at __randomizedtesting.SeedInfo.seed([F9074AED85928185:78E1C4F5F2CDE1B9]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertFalse(Assert.java:68)
	at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.doTest(ChaosMonkeyNothingIsSafeTest.java:223)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11991 lines...]
   [junit4] Suite: org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest
   [junit4]   2> Creating dataDir: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/init-core-data-001
   [junit4]   2> 808872 T4643 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /jq/b
   [junit4]   2> 808879 T4643 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> 808880 T4643 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 808881 T4644 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 808981 T4643 oasc.ZkTestServer.run start zk server on port:21306
   [junit4]   2> 808982 T4643 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 808983 T4643 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 808988 T4651 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4b0328d5 name:ZooKeeperConnection Watcher:127.0.0.1:21306 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 808988 T4643 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 808989 T4643 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 808989 T4643 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 808993 T4643 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 808993 T4643 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 808995 T4654 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14f300e7 name:ZooKeeperConnection Watcher:127.0.0.1:21306/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 808995 T4643 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 808996 T4643 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 808996 T4643 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 808999 T4643 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 809001 T4643 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 809003 T4643 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 809006 T4643 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 809006 T4643 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 809010 T4643 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 809010 T4643 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 809013 T4643 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 809014 T4643 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 809016 T4643 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 809017 T4643 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 809019 T4643 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 809020 T4643 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 809023 T4643 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 809023 T4643 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 809026 T4643 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 809026 T4643 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 809029 T4643 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 809030 T4643 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 809032 T4643 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 809033 T4643 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 809035 T4643 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 809036 T4643 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 809038 T4643 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 809039 T4643 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 810191 T4643 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 810196 T4643 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:21315
   [junit4]   2> 810196 T4643 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 810196 T4643 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 810197 T4643 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-002
   [junit4]   2> 810197 T4643 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-002/'
   [junit4]   2> 810227 T4643 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-002/solr.xml
   [junit4]   2> 810298 T4643 oasc.CoreContainer.<init> New CoreContainer 1489262136
   [junit4]   2> 810298 T4643 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-002/]
   [junit4]   2> 810299 T4643 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 810300 T4643 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 810300 T4643 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 810300 T4643 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 810300 T4643 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 810301 T4643 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 810301 T4643 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 810301 T4643 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 810301 T4643 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 810302 T4643 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 810302 T4643 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 810302 T4643 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 810303 T4643 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 810303 T4643 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 810303 T4643 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:21306/solr
   [junit4]   2> 810303 T4643 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 810304 T4643 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 810304 T4643 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 810307 T4666 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a9f45d3 name:ZooKeeperConnection Watcher:127.0.0.1:21306 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 810308 T4643 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 810308 T4643 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 810310 T4643 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 810312 T4669 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@62b92816 name:ZooKeeperConnection Watcher:127.0.0.1:21306/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 810312 T4643 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 810314 T4643 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 810316 T4643 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 810318 T4643 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 810320 T4643 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 810322 T4643 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 810325 T4643 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 810326 T4643 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:21315_jq%2Fb
   [junit4]   2> 810327 T4643 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:21315_jq%2Fb
   [junit4]   2> 810329 T4643 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 810330 T4643 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 810332 T4643 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 810333 T4643 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:21315_jq%2Fb
   [junit4]   2> 810333 T4643 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 810335 T4643 oasc.Overseer.start Overseer (id=92856226149433347-127.0.0.1:21315_jq%2Fb-n_0000000000) starting
   [junit4]   2> 810337 T4643 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 810343 T4643 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 810343 T4671 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 810344 T4643 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 810346 T4643 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 810347 T4643 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 810350 T4670 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 810354 T4673 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
   [junit4]   2> 810354 T4673 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 810355 T4673 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 810355 T4669 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 810358 T4670 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:21315/jq/b",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "node_name":"127.0.0.1:21315_jq%2Fb"}
   [junit4]   2> 810358 T4670 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
   [junit4]   2> 810358 T4670 oasc.Overseer$ClusterStateUpdater.createCollection state version control_collection 1
   [junit4]   2> 810359 T4670 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 810361 T4669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 811355 T4673 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 811356 T4673 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
   [junit4]   2> 811357 T4673 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 811357 T4673 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 811358 T4673 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 811358 T4673 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-002/collection1/'
   [junit4]   2> 811360 T4673 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-002/collection1/lib/.svn/' to classloader
   [junit4]   2> 811361 T4673 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-002/collection1/lib/README' to classloader
   [junit4]   2> 811362 T4673 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-002/collection1/lib/classes/' to classloader
   [junit4]   2> 811395 T4673 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 811398 T4673 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 811435 T4673 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 811483 T4673 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 811484 T4673 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 811502 T4673 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 811848 T4673 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 811851 T4673 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 811853 T4673 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 811857 T4673 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 811881 T4673 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 811881 T4673 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 811948 T4673 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-002/collection1/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-001/control/data/
   [junit4]   2> 811948 T4673 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@241ccc63
   [junit4]   2> 811951 T4673 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-001/control/data
   [junit4]   2> 811952 T4673 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-001/control/data/index/
   [junit4]   2> 811953 T4673 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
   [junit4]   2> 811954 T4673 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-001/control/data/index
   [junit4]   2> 811955 T4673 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=35, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.48840439297135774]
   [junit4]   2> 811960 T4673 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
   [junit4]   2> 811960 T4673 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 811964 T4673 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 811964 T4673 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 811965 T4673 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 811965 T4673 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 811965 T4673 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 811965 T4673 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 811966 T4673 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 811966 T4673 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 811966 T4673 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 811967 T4673 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 811967 T4673 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 811968 T4673 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 811968 T4673 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler
   [junit4]   2> 811969 T4673 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 811969 T4673 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 811969 T4673 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 811969 T4673 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 811970 T4673 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 811970 T4673 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 811971 T4673 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 811971 T4673 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 811990 T4673 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 811992 T4673 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 811994 T4673 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 811996 T4673 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 811999 T4673 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 812000 T4673 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 812000 T4673 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=44, maxMergeAtOnceExplicit=49, maxMergedSegmentMB=0.88671875, floorSegmentMB=0.2197265625, forceMergeDeletesPctAllowed=7.615991310789986, segmentsPerTier=13.0, maxCFSSegmentSizeMB=1.0859375, noCFSRatio=0.2402170623670979
   [junit4]   2> 812002 T4673 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
   [junit4]   2> 812002 T4673 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 812002 T4673 oass.SolrIndexSearcher.<init> Opening Searcher@6630cb9[collection1] main
   [junit4]   2> 812003 T4673 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 812004 T4673 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 812004 T4673 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 812005 T4673 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 812005 T4673 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 812005 T4673 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 812006 T4673 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 812006 T4673 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 812007 T4673 oascc.SolrZkClient.makePath makePath: /configs/conf1/_rest_managed.json
   [junit4]   2> 812009 T4673 oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new znode /configs/conf1/_rest_managed.json
   [junit4]   2> 812009 T4673 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 812009 T4673 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 812012 T4673 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 812012 T4674 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6630cb9[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 812013 T4677 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:21315/jq/b collection:control_collection shard:shard1
   [junit4]   2> 812013 T4643 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 812014 T4643 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 812014 T4677 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 812014 T4643 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 812015 T4643 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 812017 T4680 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4b2ba413 name:ZooKeeperConnection Watcher:127.0.0.1:21306/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 812017 T4643 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 812017 T4643 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 812019 T4643 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 812020 T4677 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 812022 T4669 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 812022 T4677 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 812023 T4677 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> 812023 T4643 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> ASYNC  NEW_CORE C2725 name=collection1 org.apache.solr.core.SolrCore@350ca875 url=http://127.0.0.1:21315/jq/b/collection1 node=127.0.0.1:21315_jq%2Fb C2725_STATE=coll:control_collection core:collection1 props:{base_url=http://127.0.0.1:21315/jq/b, state=down, core=collection1, node_name=127.0.0.1:21315_jq%2Fb}
   [junit4]   2> 812023 T4677 C2725 P21315 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:21315/jq/b/collection1/
   [junit4]   2> 812024 T4677 C2725 P21315 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 812024 T4677 C2725 P21315 oasc.SyncStrategy.syncToMe http://127.0.0.1:21315/jq/b/collection1/ has no replicas
   [junit4]   2> 812024 T4677 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:21315/jq/b/collection1/ shard1
   [junit4]   2> 812024 T4677 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 812025 T4670 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection"}
   [junit4]   2> 812028 T4669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 812028 T4680 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 812033 T4670 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:21315/jq/b",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"active"}
   [junit4]   2> 812036 T4669 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 812037 T4669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 812037 T4680 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 812080 T4677 oasc.ZkController.register We are http://127.0.0.1:21315/jq/b/collection1/ and leader is http://127.0.0.1:21315/jq/b/collection1/
   [junit4]   2> 812081 T4677 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:21315/jq/b
   [junit4]   2> 812081 T4677 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 812081 T4677 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
   [junit4]   2> 812081 T4677 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 812082 T4669 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 812086 T4670 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:21315/jq/b",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "node_name":"127.0.0.1:21315_jq%2Fb"}
   [junit4]   2> 812190 T4669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 812190 T4680 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 813159 T4643 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
   [junit4]   2> 813159 T4643 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 813162 T4643 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:21325
   [junit4]   2> 813162 T4643 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 813163 T4643 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 813163 T4643 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-003
   [junit4]   2> 813163 T4643 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-003/'
   [junit4]   2> 813192 T4643 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-003/solr.xml
   [junit4]   2> 813246 T4643 oasc.CoreContainer.<init> New CoreContainer 1052143585
   [junit4]   2> 813247 T4643 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-003/]
   [junit4]   2> 813247 T4643 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 813248 T4643 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 813248 T4643 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 813248 T4643 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 813248 T4643 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 813249 T4643 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 813249 T4643 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 813249 T4643 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 813249 T4643 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 813250 T4643 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 813250 T4643 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 813250 T4643 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 813251 T4643 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 813251 T4643 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 813251 T4643 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:21306/solr
   [junit4]   2> 813251 T4643 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 813251 T4643 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 813252 T4643 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 813254 T4692 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5522ee8a name:ZooKeeperConnection Watcher:127.0.0.1:21306 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 813255 T4643 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 813255 T4643 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 813257 T4643 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 813259 T4695 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6d8a95d8 name:ZooKeeperConnection Watcher:127.0.0.1:21306/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 813259 T4643 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 813265 T4643 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 814269 T4643 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:21325_jq%2Fb
   [junit4]   2> 814270 T4643 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:21325_jq%2Fb
   [junit4]   2> 814274 T4643 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 814282 T4696 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 814282 T4696 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 814283 T4669 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 814283 T4696 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 814286 T4670 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:21325/jq/b",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "node_name":"127.0.0.1:21325_jq%2Fb"}
   [junit4]   2> 814286 T4670 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1]
   [junit4]   2> 814286 T4670 oasc.Overseer$ClusterStateUpdater.createCollection state version collection1 1
   [junit4]   2> 814286 T4670 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 814289 T4669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 814289 T4695 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 814289 T4680 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 815283 T4696 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 815284 T4696 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 815285 T4696 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 815285 T4696 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 815286 T4696 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 815286 T4696 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-003/collection1/'
   [junit4]   2> 815288 T4696 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-003/collection1/lib/.svn/' to classloader
   [junit4]   2> 815288 T4696 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-003/collection1/lib/README' to classloader
   [junit4]   2> 815289 T4696 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-003/collection1/lib/classes/' to classloader
   [junit4]   2> 815321 T4696 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 815324 T4696 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 815360 T4696 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 815408 T4696 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 815409 T4696 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 815423 T4696 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 815766 T4696 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 815769 T4696 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 815771 T4696 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 815776 T4696 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 815799 T4696 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 815799 T4696 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 815799 T4696 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-003/collection1/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-001/jetty1/
   [junit4]   2> 815800 T4696 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@241ccc63
   [junit4]   2> 815801 T4696 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-001/jetty1
   [junit4]   2> 815802 T4696 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-001/jetty1/index/
   [junit4]   2> 815803 T4696 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-001/jetty1/index' doesn't exist. Creating new index...
   [junit4]   2> 815804 T4696 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-001/jetty1/index
   [junit4]   2> 815805 T4696 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=35, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.48840439297135774]
   [junit4]   2> 815810 T4696 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
   [junit4]   2> 815810 T4696 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 815814 T4696 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 815815 T4696 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 815815 T4696 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 815815 T4696 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 815815 T4696 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 815816 T4696 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 815816 T4696 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 815816 T4696 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 815817 T4696 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 815817 T4696 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 815818 T4696 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 815818 T4696 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 815819 T4696 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler
   [junit4]   2> 815819 T4696 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 815820 T4696 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 815820 T4696 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 815821 T4696 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 815821 T4696 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 815822 T4696 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 815822 T4696 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 815823 T4696 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 815841 T4696 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 815843 T4696 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 815844 T4696 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 815846 T4696 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 815850 T4696 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 815850 T4696 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 815851 T4696 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=44, maxMergeAtOnceExplicit=49, maxMergedSegmentMB=0.88671875, floorSegmentMB=0.2197265625, forceMergeDeletesPctAllowed=7.615991310789986, segmentsPerTier=13.0, maxCFSSegmentSizeMB=1.0859375, noCFSRatio=0.2402170623670979
   [junit4]   2> 815853 T4696 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
   [junit4]   2> 815853 T4696 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 815854 T4696 oass.SolrIndexSearcher.<init> Opening Searcher@604bada2[collection1] main
   [junit4]   2> 815854 T4696 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 815855 T4696 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 815855 T4696 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 815856 T4696 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 815856 T4696 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 815856 T4696 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 815857 T4696 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
   [junit4]   2> 815857 T4696 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 815858 T4696 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
   [junit4]   2> 815858 T4696 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 815862 T4697 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@604bada2[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 815862 T4696 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 815863 T4700 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:21325/jq/b collection:collection1 shard:shard1
   [junit4]   2> 815863 T4643 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 815864 T4643 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 815864 T4700 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 815869 T4700 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 815871 T4669 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 815871 T4700 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 815871 T4700 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C2726 name=collection1 org.apache.solr.core.SolrCore@79f69d3f url=http://127.0.0.1:21325/jq/b/collection1 node=127.0.0.1:21325_jq%2Fb C2726_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:21325/jq/b, state=down, core=collection1, node_name=127.0.0.1:21325_jq%2Fb}
   [junit4]   2> 815872 T4700 C2726 P21325 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:21325/jq/b/collection1/
   [junit4]   2> 815872 T4700 C2726 P21325 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 815872 T4700 C2726 P21325 oasc.SyncStrategy.syncToMe http://127.0.0.1:21325/jq/b/collection1/ has no replicas
   [junit4]   2> 815872 T4700 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:21325/jq/b/collection1/ shard1
   [junit4]   2> 815873 T4700 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 815874 T4670 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"collection1"}
   [junit4]   2> 815877 T4695 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 815877 T4680 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 815877 T4669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 815882 T4670 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:21325/jq/b",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"active"}
   [junit4]   2> 815884 T4669 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 815885 T4669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 815885 T4695 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 815885 T4680 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 815928 T4700 oasc.ZkController.register We are http://127.0.0.1:21325/jq/b/collection1/ and leader is http://127.0.0.1:21325/jq/b/collection1/
   [junit4]   2> 815929 T4700 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:21325/jq/b
   [junit4]   2> 815929 T4700 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 815929 T4700 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
   [junit4]   2> 815929 T4700 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 815930 T4669 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 815933 T4670 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:21325/jq/b",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "node_name":"127.0.0.1:21325_jq%2Fb"}
   [junit4]   2> 816037 T4669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 816037 T4695 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 816037 T4680 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 817065 T4643 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
   [junit4]   2> 817065 T4643 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 817068 T4643 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:21329
   [junit4]   2> 817069 T4643 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 817069 T4643 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 817069 T4643 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-004
   [junit4]   2> 817069 T4643 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-004/'
   [junit4]   2> 817099 T4643 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-004/solr.xml
   [junit4]   2> 817151 T4643 oasc.CoreContainer.<init> New CoreContainer 239422443
   [junit4]   2> 817151 T4643 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-004/]
   [junit4]   2> 817152 T4643 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 817152 T4643 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 817153 T4643 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 817153 T4643 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 817153 T4643 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 817153 T4643 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 817153 T4643 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 817154 T4643 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 817154 T4643 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 817154 T4643 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 817155 T4643 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 817155 T4643 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 817155 T4643 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 817155 T4643 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 817156 T4643 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:21306/solr
   [junit4]   2> 817156 T4643 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 817156 T4643 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 817157 T4643 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 817159 T4712 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@24e3a784 name:ZooKeeperConnection Watcher:127.0.0.1:21306 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 817160 T4643 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 817160 T4643 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 817162 T4643 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 817163 T4715 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e3e4b38 name:ZooKeeperConnection Watcher:127.0.0.1:21306/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 817163 T4643 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 817169 T4643 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 818174 T4643 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:21329_jq%2Fb
   [junit4]   2> 818175 T4643 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:21329_jq%2Fb
   [junit4]   2> 818180 T4643 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 818186 T4716 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 818187 T4716 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 818188 T4669 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 818188 T4716 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 818191 T4670 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:21329/jq/b",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "node_name":"127.0.0.1:21329_jq%2Fb"}
   [junit4]   2> 818191 T4670 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
   [junit4]   2> 818191 T4670 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 818194 T4669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 818194 T4695 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 818194 T4715 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 818194 T4680 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 819188 T4716 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 819189 T4716 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 819190 T4716 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 819190 T4716 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 819191 T4716 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 819191 T4716 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-004/collection1/'
   [junit4]   2> 819193 T4716 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-004/collection1/lib/README' to classloader
   [junit4]   2> 819193 T4716 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-004/collection1/lib/classes/' to classloader
   [junit4]   2> 819194 T4716 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-004/collection1/lib/.svn/' to classloader
   [junit4]   2> 819227 T4716 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 819229 T4716 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 819278 T4716 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 819324 T4716 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 819326 T4716 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 819340 T4716 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 819673 T4716 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 819676 T4716 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 819678 T4716 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 819682 T4716 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 819706 T4716 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 819706 T4716 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 819706 T4716 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-004/collection1/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-001/jetty2/
   [junit4]   2> 819707 T4716 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@241ccc63
   [junit4]   2> 819708 T4716 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-001/jetty2
   [junit4]   2> 819709 T4716 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-001/jetty2/index/
   [junit4]   2> 819710 T4716 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-001/jetty2/index' doesn't exist. Creating new index...
   [junit4]   2> 819711 T4716 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-001/jetty2/index
   [junit4]   2> 819712 T4716 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=35, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.48840439297135774]
   [junit4]   2> 819720 T4716 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-001/jetty2/index,segFN=segments_1,generation=1}
   [junit4]   2> 819720 T4716 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 819724 T4716 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 819725 T4716 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 819725 T4716 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 819725 T4716 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 819726 T4716 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 819726 T4716 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 819726 T4716 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 819727 T4716 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 819727 T4716 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 819728 T4716 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 819728 T4716 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 819729 T4716 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 819729 T4716 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler
   [junit4]   2> 819730 T4716 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 819730 T4716 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 819730 T4716 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 819731 T4716 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 819731 T4716 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 819731 T4716 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 819732 T4716 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 819733 T4716 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 819751 T4716 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 819753 T4716 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 819754 T4716 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 819756 T4716 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 819760 T4716 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 819760 T4716 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 819761 T4716 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=44, maxMergeAtOnceExplicit=49, maxMergedSegmentMB=0.88671875, floorSegmentMB=0.2197265625, forceMergeDeletesPctAllowed=7.615991310789986, segmentsPerTier=13.0, maxCFSSegmentSizeMB=1.0859375, noCFSRatio=0.2402170623670979
   [junit4]   2> 819763 T4716 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-001/jetty2/index,segFN=segments_1,generation=1}
   [junit4]   2> 819763 T4716 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 819763 T4716 oass.SolrIndexSearcher.<init> Opening Searcher@7f98a828[collection1] main
   [junit4]   2> 819764 T4716 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 819765 T4716 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 819765 T4716 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 819765 T4716 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 819766 T4716 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 819766 T4716 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 819767 T4716 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
   [junit4]   2> 819767 T4716 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 819767 T4716 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
   [junit4]   2> 819768 T4716 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 819771 T4717 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7f98a828[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 819771 T4716 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 819772 T4720 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:21329/jq/b collection:collection1 shard:shard1
   [junit4]   2> 819773 T4643 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 819773 T4643 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 819775 T4720 oasc.ZkController.register We are http://127.0.0.1:21329/jq/b/collection1/ and leader is http://127.0.0.1:21325/jq/b/collection1/
   [junit4]   2> 819775 T4720 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:21329/jq/b
   [junit4]   2> 819775 T4720 oasc.ZkController.checkRecovery Core needs to recover:collection1
   [junit4]   2> 819776 T4720 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
   [junit4]   2> ASYNC  NEW_CORE C2727 name=collection1 org.apache.solr.core.SolrCore@56ca1411 url=http://127.0.0.1:21329/jq/b/collection1 node=127.0.0.1:21329_jq%2Fb C2727_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:21329/jq/b, state=down, core=collection1, node_name=127.0.0.1:21329_jq%2Fb}
   [junit4]   2> 819776 T4721 C2727 P21329 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
   [junit4]   2> 819777 T4721 C2727 P21329 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
   [junit4]   2> 819777 T4721 C2727 P21329 oasc.RecoveryStrategy.doRecovery Publishing state of core collection1 as recovering, leader is http://127.0.0.1:21325/jq/b/collection1/ and I am http://127.0.0.1:21329/jq/b/collection1/
   [junit4]   2> 819777 T4721 C2727 P21329 oasc.ZkController.publish publishing core=collection1 state=recovering collection=collection1
   [junit4]   2> 819777 T4721 C2727 P21329 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 819779 T4669 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 819779 T4721 C2727 P21329 oasc.RecoveryStrategy.sendPrepRecoveryCmd Sending prep recovery command to http://127.0.0.1:21325/jq/b; WaitForState: action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1%3A21329_jq%252Fb&coreNodeName=core_node2&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true
   [junit4]   2> 819781 T4683 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node2, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
   [junit4]   2> 819782 T4670 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:21329/jq/b",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "state":"recovering",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node2",
   [junit4]   2> 	  "node_name":"127.0.0.1:21329_jq%2Fb"}
   [junit4]   2> 819783 T4683 oasha.CoreAdminHandler.handleWaitForStateAction Will wait a max of 183 seconds to see collection1 (shard1 of collection1) have state: recovering
   [junit4]   2> 819784 T4683 oasha.CoreAdminHandler.handleWaitForStateAction In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:21329_jq%2Fb, coreNodeName=core_node2, onlyIfActiveCheckResult=false, nodeProps: core_node2:{"base_url":"http://127.0.0.1:21329/jq/b","state":"down","core":"collection1","node_name":"127.0.0.1:21329_jq%2Fb"}
   [junit4]   2> 819785 T4669 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 819785 T4715 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 819785 T4680 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 819785 T4695 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 820784 T4683 oasha.CoreAdminHandler.handleWaitForStateAction In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:21329_jq%2Fb, coreNodeName=core_node2, onlyIfActiveCheckResult=false, nodeProps: core_node2:{"base_url":"http://127.0.0.1:21329/jq/b","state":"recovering","core":"collection1","node_name":"127.0.0.1:21329_jq%2Fb"}
   [junit4]   2> 820785 T4683 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node2, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
   [junit4]   2> 820785 T4683 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={wt=javabin&checkLive=true&state=recovering&action=PREPRECOVERY&version=2&onlyIfLeaderActive=true&core=collection1&nodeName=127.0.0.1:21329_jq%252Fb&coreNodeName=core_node2&onlyIfLeader=true} status=0 QTime=1004 
   [junit4]   2> 820995 T4643 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
   [junit4]   2> 820996 T4643 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 820998 T4643 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:21336
   [junit4]   2> 820999 T4643 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 820999 T4643 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 820999 T4643 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-005
   [junit4]   2> 821000 T4643 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-005/'
   [junit4]   2> 821029 T4643 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-005/solr.xml
   [junit4]   2> 821081 T4643 oasc.CoreContainer.<init> New CoreContainer 1110720612
   [junit4]   2> 821081 T4643 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-005/]
   [junit4]   2> 821082 T4643 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 821082 T4643 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 821082 T4643 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 821083 T4643 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 821083 T4643 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 821083 T4643 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 821083 T4643 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 821083 T4643 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 821084 T4643 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 821084 T4643 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 821084 T4643 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 821085 T46

[...truncated too long message...]

 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 943418 T4643 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/jq/b,null}
   [junit4]   2> 948421 T4643 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 21336
   [junit4]   2> 948422 T4643 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1976519436
   [junit4]   2> 948422 T4643 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 948422 T4643 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 948424 T4868 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 948424 T4643 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@22d58575
   [junit4]   2> 948425 T4643 oasc.ZkController.unRegisterConfListener  a listener was removed because of core close
   [junit4]   2> 948425 T4643 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=3,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=73,cumulative_deletesById=28,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=8368,transaction_logs_total_number=1}
   [junit4]   2> 948426 T4643 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 948426 T4643 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 948426 T4643 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 948427 T4919 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:21336/jq/b",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node3",
   [junit4]   2> 	  "node_name":"127.0.0.1:21336_jq%2Fb"}
   [junit4]   2> 948429 T4868 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 948458 T4643 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 948463 T4643 oasc.CachingDirectoryFactory.close Closing StandardDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 948463 T4643 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-001/jetty3/index.20141125000124584 [CachedDir<<refCount=0;path=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-001/jetty3/index.20141125000124584;done=false>>]
   [junit4]   2> 948463 T4643 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-001/jetty3/index.20141125000124584
   [junit4]   2> 948463 T4643 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-001/jetty3 [CachedDir<<refCount=0;path=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-001/jetty3;done=false>>]
   [junit4]   2> 948464 T4643 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001/tempDir-001/jetty3
   [junit4]   2> 948464 T4643 oasc.Overseer.close Overseer (id=92856226149433362-127.0.0.1:21336_jq%2Fb-n_0000000007) closing
   [junit4]   2> 948464 T4919 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:21336_jq%2Fb
   [junit4]   2> 948466 T4868 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 948467 T4643 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/jq/b,null}
   [junit4]   2> 953469 T4643 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
   [junit4]   2> 953471 T4643 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:21306 21306
   [junit4]   2> 953697 T4644 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:21306 21306
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ChaosMonkeyNothingIsSafeTest -Dtests.method=testDistribSearch -Dtests.seed=F9074AED85928185 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=hr_HR -Dtests.timezone=Europe/Guernsey -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] FAILURE  145s J0 | ChaosMonkeyNothingIsSafeTest.testDistribSearch <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: There were too many update fails - we expect it can happen, but shouldn't easily
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F9074AED85928185:78E1C4F5F2CDE1B9]:0)
   [junit4]    > 	at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.doTest(ChaosMonkeyNothingIsSafeTest.java:223)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 953708 T4643 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-F9074AED85928185-001
   [junit4]   2> 144840 T4642 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene50): {a_t=FSTOrd50, _version_=PostingsFormat(name=LuceneFixedGap), a_i=PostingsFormat(name=Memory doPackFST= false), rnd_b=PostingsFormat(name=Memory doPackFST= false), id=PostingsFormat(name=Memory doPackFST= false)}, docValues:{}, sim=DefaultSimilarity, locale=hr_HR, timezone=Europe/Guernsey
   [junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_65 (64-bit)/cpus=16,threads=1,free=131616448,total=385351680
   [junit4]   2> NOTE: All tests run in this JVM: [TermsComponentTest, CacheHeaderTest, TestChildDocTransformer, TestLazyCores, TestSolrDeletionPolicy1, AlternateDirectoryTest, ConvertedLegacyTest, TestFiltering, ZkSolrClientTest, TestTrie, TestRangeQuery, TestImplicitCoreProperties, SynonymTokenizerTest, CloudExitableDirectoryReaderTest, TestWordDelimiterFilterFactory, AtomicUpdatesTest, TestCloudManagedSchemaConcurrent, DistributedExpandComponentTest, DistributedSpellCheckComponentTest, HdfsDirectoryTest, SpellingQueryConverterTest, TestDefaultSimilarityFactory, TestManagedSchemaFieldResource, TestManagedSchemaFieldTypeResource, TestStressReorder, SharedFSAutoReplicaFailoverUtilsTest, CoreAdminRequestStatusTest, TestXIncludeConfig, TestRandomFaceting, TestExtendedDismaxParser, MoreLikeThisHandlerTest, JsonLoaderTest, SimplePostToolTest, TestUtils, TestSolrXml, HighlighterMaxOffsetTest, SolrTestCaseJ4Test, SSLMigrationTest, DirectSolrConnectionTest, CoreMergeIndexesAdminHandlerTest, TestSolrDeletionPolicy2, TestIBSimilarityFactory, TestReplicationHandlerBackup, ChaosMonkeyNothingIsSafeTest]
   [junit4] Completed on J0 in 145.13s, 1 test, 1 failure <<< FAILURES!

[...truncated 1037 lines...]
BUILD FAILED
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:532: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:473: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:61: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/extra-targets.xml:39: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:508: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1363: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:970: There were test failures: 449 suites, 1825 tests, 1 failure, 55 ignored (9 assumptions)

Total time: 194 minutes 21 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #680
Archived 5 artifacts
Archive block size is 32768
Received 0 blocks and 829085228 bytes
Compression is 0.0%
Took 2 min 23 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 682 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/682/

3 tests failed.
FAILED:  org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([80F928B7C834E5E4]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([80F928B7C834E5E4]:0)


FAILED:  org.apache.solr.handler.component.DistributedFacetPivotSmallTest.testDistribSearch

Error Message:
Captured an uncaught exception in thread: Thread[id=397, name=Thread-252, state=RUNNABLE, group=TGRP-DistributedFacetPivotSmallTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=397, name=Thread-252, state=RUNNABLE, group=TGRP-DistributedFacetPivotSmallTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:63284/_o/ub, http://[ff01::083]:33332/_o/ub, http://[ff01::213]:33332/_o/ub, http://127.0.0.1:63296/_o/ub, http://[ff01::114]:33332/_o/ub]
	at __randomizedtesting.SeedInfo.seed([80F928B7C834E5E4]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:569)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:215)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:211)
	at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:91)
	at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:301)
	at org.apache.solr.BaseDistributedSearchTestCase$5.run(BaseDistributedSearchTestCase.java:575)




Build Log:
[...truncated 11560 lines...]
   [junit4] Suite: org.apache.solr.handler.component.DistributedFacetPivotSmallTest
   [junit4]   2> Creating dataDir: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.component.DistributedFacetPivotSmallTest-80F928B7C834E5E4-001/init-core-data-001
   [junit4]   2> 102428 T269 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 102429 T269 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /_o/ub
   [junit4]   2> 102434 T269 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> 102438 T269 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 102443 T269 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:63282
   [junit4]   2> 102444 T269 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 102444 T269 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 102445 T269 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr
   [junit4]   2> 102445 T269 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/'
   [junit4]   2> 102502 T269 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/solr.xml
   [junit4]   2> 102621 T269 oasc.CoreContainer.<init> New CoreContainer 1541002680
   [junit4]   2> 102622 T269 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/]
   [junit4]   2> 102623 T269 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 102624 T269 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 102624 T269 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 102625 T269 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 102625 T269 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 102626 T269 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 102626 T269 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 102626 T269 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 102627 T269 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 102627 T269 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 102628 T269 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 102629 T269 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 102629 T269 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 102630 T269 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 102636 T279 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 102638 T279 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 102639 T279 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 102640 T279 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 102732 T279 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 102819 T279 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 102821 T279 oass.IndexSchema.readSchema Reading Solr Schema from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml
   [junit4]   2> 102839 T279 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 103306 T279 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 103324 T279 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 103328 T279 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 103347 T279 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 103354 T279 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 103359 T279 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 103360 T279 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 103361 T279 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 103361 T279 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 103362 T279 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 103362 T279 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 103363 T279 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from instancedir /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/
   [junit4]   2> 103363 T279 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 103364 T279 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.component.DistributedFacetPivotSmallTest-80F928B7C834E5E4-001/tempDir-001/control/data/
   [junit4]   2> 103364 T279 oasc.JmxMonitoredMap.<init> No JMX servers found, not exposing Solr information with JMX.
   [junit4]   2> 103365 T279 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.component.DistributedFacetPivotSmallTest-80F928B7C834E5E4-001/tempDir-001/control/data
   [junit4]   2> 103365 T279 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.component.DistributedFacetPivotSmallTest-80F928B7C834E5E4-001/tempDir-001/control/data/index/
   [junit4]   2> 103366 T279 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.component.DistributedFacetPivotSmallTest-80F928B7C834E5E4-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
   [junit4]   2> 103366 T279 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.component.DistributedFacetPivotSmallTest-80F928B7C834E5E4-001/tempDir-001/control/data/index
   [junit4]   2> 103367 T279 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=17, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 103368 T279 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@5663489e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1034a961; maxCacheMB=0.5673828125 maxMergeSizeMB=0.9296875))),segFN=segments_1,generation=1}
   [junit4]   2> 103368 T279 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 103369 T279 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
   [junit4]   2> 103371 T279 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 103371 T279 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 103371 T279 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 103372 T279 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 103372 T279 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 103373 T279 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 103373 T279 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 103374 T279 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 103374 T279 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 103375 T279 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 103375 T279 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 103376 T279 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 103376 T279 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 103377 T279 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 103377 T279 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 103377 T279 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 103378 T279 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 103378 T279 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 103378 T279 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 103379 T279 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
   [junit4]   2> 103384 T279 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
   [junit4]   2> 103385 T279 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 103385 T279 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 103386 T279 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 103386 T279 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
   [junit4]   2> 103387 T279 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
   [junit4]   2> 103387 T279 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
   [junit4]   2> 103387 T279 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 103388 T279 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 103388 T279 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 103388 T279 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 103389 T279 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 103389 T279 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 103389 T279 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 103390 T279 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 103390 T279 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 103391 T279 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
   [junit4]   2> 103391 T279 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
   [junit4]   2> 103423 T279 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 103425 T279 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 103428 T279 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 103431 T279 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 103433 T279 oasc.SolrCore.initDeprecatedSupport WARN solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
   [junit4]   2> 103435 T279 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 103435 T279 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 103436 T279 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=35, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 103437 T279 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@5663489e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1034a961; maxCacheMB=0.5673828125 maxMergeSizeMB=0.9296875))),segFN=segments_1,generation=1}
   [junit4]   2> 103437 T279 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 103437 T279 oass.SolrIndexSearcher.<init> Opening Searcher@2f17b946[collection1] main
   [junit4]   2> 103438 T279 oasr.RestManager.init Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
   [junit4]   2> 103438 T279 oasr.ManagedResourceStorage.load Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 103439 T279 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /rest/managed
   [junit4]   2> 103439 T279 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using InMemoryStorage
   [junit4]   2> 103439 T279 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 103440 T279 oashc.SpellCheckComponent.inform Initializing spell checkers
   [junit4]   2> 103457 T279 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 103528 T280 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2f17b946[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 103528 T279 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 103529 T269 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 103530 T269 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 103533 T269 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 103536 T269 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:63284
   [junit4]   2> 103537 T269 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 103537 T269 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 103538 T269 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr
   [junit4]   2> 103538 T269 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/'
   [junit4]   2> 103609 T269 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/solr.xml
   [junit4]   2> 103738 T269 oasc.CoreContainer.<init> New CoreContainer 326164043
   [junit4]   2> 103739 T269 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/]
   [junit4]   2> 103741 T269 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 103741 T269 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 103741 T269 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 103742 T269 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 103742 T269 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 103743 T269 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 103743 T269 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 103743 T269 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 103744 T269 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 103744 T269 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 103745 T269 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 103746 T269 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 103746 T269 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 103747 T269 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 103753 T291 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 103755 T291 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 103756 T291 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 103757 T291 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 103852 T291 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 103938 T291 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 103940 T291 oass.IndexSchema.readSchema Reading Solr Schema from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml
   [junit4]   2> 103956 T291 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 104438 T291 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 104458 T291 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 104461 T291 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 104482 T291 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 104488 T291 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 104493 T291 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 104495 T291 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 104495 T291 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 104495 T291 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 104496 T291 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 104497 T291 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 104497 T291 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from instancedir /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/
   [junit4]   2> 104497 T291 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 104498 T291 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.component.DistributedFacetPivotSmallTest-80F928B7C834E5E4-001/tempDir-001/shard0/data/
   [junit4]   2> 104498 T291 oasc.JmxMonitoredMap.<init> No JMX servers found, not exposing Solr information with JMX.
   [junit4]   2> 104499 T291 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.component.DistributedFacetPivotSmallTest-80F928B7C834E5E4-001/tempDir-001/shard0/data
   [junit4]   2> 104499 T291 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.component.DistributedFacetPivotSmallTest-80F928B7C834E5E4-001/tempDir-001/shard0/data/index/
   [junit4]   2> 104500 T291 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.component.DistributedFacetPivotSmallTest-80F928B7C834E5E4-001/tempDir-001/shard0/data/index' doesn't exist. Creating new index...
   [junit4]   2> 104500 T291 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.component.DistributedFacetPivotSmallTest-80F928B7C834E5E4-001/tempDir-001/shard0/data/index
   [junit4]   2> 104501 T291 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=17, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 104502 T291 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@7bf6a4c2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7dbafc22; maxCacheMB=0.5673828125 maxMergeSizeMB=0.9296875))),segFN=segments_1,generation=1}
   [junit4]   2> 104502 T291 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 104503 T291 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
   [junit4]   2> 104504 T291 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 104505 T291 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 104505 T291 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 104505 T291 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 104506 T291 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 104506 T291 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 104506 T291 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 104507 T291 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 104507 T291 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 104508 T291 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 104509 T291 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 104509 T291 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 104510 T291 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 104510 T291 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 104510 T291 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 104511 T291 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 104511 T291 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 104511 T291 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 104512 T291 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 104512 T291 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
   [junit4]   2> 104512 T291 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
   [junit4]   2> 104513 T291 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 104513 T291 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 104514 T291 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 104514 T291 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
   [junit4]   2> 104514 T291 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
   [junit4]   2> 104514 T291 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
   [junit4]   2> 104515 T291 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 104515 T291 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 104516 T291 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 104516 T291 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 104516 T291 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 104517 T291 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 104517 T291 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 104517 T291 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 104518 T291 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 104518 T291 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
   [junit4]   2> 104518 T291 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
   [junit4]   2> 104548 T291 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 104550 T291 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 104553 T291 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 104555 T291 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 104557 T291 oasc.SolrCore.initDeprecatedSupport WARN solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
   [junit4]   2> 104559 T291 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 104560 T291 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 104560 T291 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=35, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 104561 T291 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@7bf6a4c2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7dbafc22; maxCacheMB=0.5673828125 maxMergeSizeMB=0.9296875))),segFN=segments_1,generation=1}
   [junit4]   2> 104562 T291 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 104562 T291 oass.SolrIndexSearcher.<init> Opening Searcher@193d3cf9[collection1] main
   [junit4]   2> 104563 T291 oasr.RestManager.init Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
   [junit4]   2> 104563 T291 oasr.ManagedResourceStorage.load Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 104563 T291 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /rest/managed
   [junit4]   2> 104564 T291 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using InMemoryStorage
   [junit4]   2> 104564 T291 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 104564 T291 oashc.SpellCheckComponent.inform Initializing spell checkers
   [junit4]   2> 104613 T291 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 104824 T291 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 104824 T292 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@193d3cf9[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 104825 T269 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 104825 T269 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 104828 T269 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 104831 T269 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:63288
   [junit4]   2> 104832 T269 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 104832 T269 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 104832 T269 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr
   [junit4]   2> 104832 T269 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/'
   [junit4]   2> 104871 T269 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/solr.xml
   [junit4]   2> 104935 T269 oasc.CoreContainer.<init> New CoreContainer 1267111885
   [junit4]   2> 104935 T269 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/]
   [junit4]   2> 104937 T269 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 104937 T269 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 104937 T269 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 104937 T269 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 104938 T269 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 104938 T269 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 104938 T269 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 104938 T269 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 104939 T269 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 104939 T269 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 104940 T269 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 104940 T269 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 104940 T269 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 104941 T269 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 104945 T303 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 104947 T303 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 104947 T303 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 104947 T303 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 105003 T303 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 105058 T303 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 105059 T303 oass.IndexSchema.readSchema Reading Solr Schema from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml
   [junit4]   2> 105070 T303 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 105512 T303 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 105524 T303 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 105526 T303 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 105539 T303 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 105543 T303 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 105546 T303 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 105547 T303 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 105547 T303 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 105547 T303 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 105548 T303 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 105548 T303 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 105548 T303 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from instancedir /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/
   [junit4]   2> 105549 T303 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 105549 T303 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.component.DistributedFacetPivotSmallTest-80F928B7C834E5E4-001/tempDir-001/shard1/data/
   [junit4]   2> 105549 T303 oasc.JmxMonitoredMap.<init> No JMX servers found, not exposing Solr information with JMX.
   [junit4]   2> 105550 T303 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.component.DistributedFacetPivotSmallTest-80F928B7C834E5E4-001/tempDir-001/shard1/data
   [junit4]   2> 105550 T303 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.component.DistributedFacetPivotSmallTest-80F928B7C834E5E4-001/tempDir-001/shard1/data/index/
   [junit4]   2> 105550 T303 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.component.DistributedFacetPivotSmallTest-80F928B7C834E5E4-001/tempDir-001/shard1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 105551 T303 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.component.DistributedFacetPivotSmallTest-80F928B7C834E5E4-001/tempDir-001/shard1/data/index
   [junit4]   2> 105551 T303 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=17, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 105552 T303 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@76cf4c92 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@23ac422e; maxCacheMB=0.5673828125 maxMergeSizeMB=0.9296875))),segFN=segments_1,generation=1}
   [junit4]   2> 105552 T303 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 105552 T303 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
   [junit4]   2> 105554 T303 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 105554 T303 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 105554 T303 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 105554 T303 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 105555 T303 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 105555 T303 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 105555 T303 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 105555 T303 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 105555 T303 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 105556 T303 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 105556 T303 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 105556 T303 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 105557 T303 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 105557 T303 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 105557 T303 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 105557 T303 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 105558 T303 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 105558 T303 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 105558 T303 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 105558 T303 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
   [junit4]   2> 105559 T303 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
   [junit4]   2> 105559 T303 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 105559 T303 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 105559 T303 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 105560 T303 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
   [junit4]   2> 105560 T303 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
   [junit4]   2> 105560 T303 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
   [junit4]   2> 105560 T303 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 105561 T303 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 105561 T303 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 105561 T303 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 105561 T303 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 105561 T303 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 105562 T303 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 105562 T303 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 105562 T303 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 105562 T303 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
   [junit4]   2> 105563 T303 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
   [junit4]   2> 105578 T303 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 105580 T303 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 105581 T303 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 105583 T303 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 105584 T303 oasc.SolrCore.initDeprecatedSupport WARN solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
   [junit4]   2> 105586 T303 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 105586 T303 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 105586 T303 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=35, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 105587 T303 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@76cf4c92 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@23ac422e; maxCacheMB=0.5673828125 maxMergeSizeMB=0.9296875))),segFN=segments_1,generation=1}
   [junit4]   2> 105587 T303 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 105588 T303 oass.SolrIndexSearcher.<init> Opening Searcher@6ea1bf09[collection1] main
   [junit4]   2> 105588 T303 oasr.RestManager.init Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
   [junit4]   2> 105588 T303 oasr.ManagedResourceStorage.load Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 105588 T303 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /rest/managed
   [junit4]   2> 105589 T303 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using InMemoryStorage
   [junit4]   2> 105589 T303 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 105589 T303 oashc.SpellCheckComponent.inform Initializing spell checkers
   [junit4]   2> 105599 T303 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 105665 T304 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6ea1bf09[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 105665 T303 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 105665 T269 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 105666 T269 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 105668 T269 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 105670 T269 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:63296
   [junit4]   2> 105670 T269 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 105671 T269 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 105671 T269 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr
   [junit4]   2> 105671 T269 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/'
   [junit4]   2> 105707 T269 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/solr.xml
   [junit4]   2> 105776 T269 oasc.CoreContainer.<init> New CoreContainer 1940200024
   [junit4]   2> 105776 T269 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/]
   [junit4]   2> 105777 T269 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 105778 T269 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 105778 T269 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 105778 T269 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 105778 T269 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 105779 T269 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 105779 T269 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 105779 T269 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 105779 T269 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 105780 T269 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 105780 T269 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 105781 T269 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 105781 T269 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 105781 T269 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 105785 T315 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 105787 T315 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 105787 T315 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 105788 T315 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 105842 T315 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 105895 T315 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 105895 T315 oass.IndexSchema.readSchema Reading Solr Schema from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml
   [junit4]   2> 105905 T315 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 106267 T315 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 106281 T315 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 106283 T315 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 106296 T315 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 106300 T315 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 106303 T315 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 106304 T315 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 106305 T315 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 106305 T315 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 106305 T315 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 106305 T315 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 106306 T315 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from instancedir /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/
   [junit4]   2> 106306 T315 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 106306 T315 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.component.DistributedFacetPivotSmallTest-80F928B7C834E5E4-001/tempDir-001/shard2/data/
   [junit4]   2> 106307 T315 oasc.JmxMonitoredMap.<init> No JMX servers found, not exposing Solr information with JMX.
   [junit4]   2> 106307 T315 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.component.DistributedFacetPivotSmallTest-80F928B7C834E5E4-001/tempDir-001/shard2/data
   [junit4]   2> 106307 T315 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.component.DistributedFacetPivotSmallTest-80F928B7C834E5E4-001/tempDir-001/shard2/data/index/
   [junit4]   2> 106307 T315 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.component.DistributedFacetPivotSmallTest-80F928B7C834E5E4-001/tempDir-001/shard2/data/index' doesn't exist. Creating new index...
   [junit4]   2> 106308 T315 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.component.DistributedFacetPivotSmallTest-80F928B7C834E5E4-001/tempDir-001/shard2/data/index
   [junit4]   2> 106308 T315 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=17, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 106309 T315 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@14fda3ee lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@18156906; maxCacheMB=0.5673828125 maxMergeSizeMB=0.9296875))),segFN=segments_1,generation=1}
   [junit4]   2> 106309 T315 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 106310 T315 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
   [junit4]   2> 106311 T315 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 106311 T315 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 106312 T315 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 106312 T315 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 106312 T315 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 106312 T315 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 106313 T315 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 106313 T315 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 106313 T315 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 106314 T315 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 106314 T315 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 106314 T315 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 106314 T315 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 106315 T315 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 106315 T315 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 106315 T315 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 106315 T315 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 106316 T315 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 106316 T315 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 106316 T315 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
   [junit4]   2> 106316 T315 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
   [junit4]   2> 106317 T315 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 106317 T315 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 106317 T315 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 106317 T315 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
   [junit4]   2> 106318 T315 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
   [junit4]   2> 106318 T315 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
   [junit4]   2> 106318 T315 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 106318 T315 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 106318 T315 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 106319 T315 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 106319 T315 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 106319 T315 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 106319 T315 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 106320 T315 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 106320 T315 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 106320 T315 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
   [junit4]   2> 106320 T315 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
   [junit4]   2> 106333 T315 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 106334 T315 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 106336 T315 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 106337 T315 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 106338 T315 oasc.SolrCore.initDeprecatedSupport WARN solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
   [junit4]   2> 106340 T315 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 106340 T315 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 106340 T315 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=35, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 106341 T315 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@14fda3ee lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@18156906; maxCacheMB=0.5673828125 maxMergeSizeMB=0.9296875))),segFN=segments_1,generation=1}
   [junit4]   2> 106341 T315 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 106342 T315 oass.SolrIndexSearcher.<init> Opening Searcher@687c8cac[collection1] main
   [junit4]   2> 106342 T315 oasr.RestManager.init Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
   [junit4]   2> 106342 T315 oasr.ManagedResourceStorage.load Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 106342 T315 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /rest/managed
   [junit4]   2> 106343 T315 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using InMemoryStorage
   [junit4]   2> 106343 T315 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 106343 T315 oashc.SpellCheckComponent.inform Initializing spell checkers
   [junit4]   2> 106380 T315 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 106430 T316 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@687c8cac[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 106431 T315 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 106431 T269 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 106432 T269 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 106434 T269 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 106436 T269 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:63297
   [junit4]   2> 106436 T269 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 106437 T269 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 106437 T269 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr
   [junit4]   2> 106437 T269 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/'
   [junit4]   2> 106468 T269 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/solr.xml
   [junit4]   2> 106538 T269 oasc.CoreContainer.<init> New CoreContainer 522992192
   [junit4]   2> 106538 T269 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/]
   [junit4]   2> 106539 T269 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 106540 T269 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 106540 T269 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 106540 T269 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 106540 T269 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 106541 T269 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 106541 T269 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 106541 T269 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 106542 T269 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 106542 T269 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 106542 T269 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 106543 T269 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 106543 T269 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 106543 T269 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 106548 T327 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 106549 T327 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 106550 T327 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 106550 T327 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 106604 T327 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 106654 T327 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 106655 T327 oass.IndexSchema.readSchema Reading Solr Schema from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml
   [junit4]   2> 106665 T327 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 107070 T327 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 107088 T327 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 107092 T327 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 107112 T327 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 107117 T327 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 107122 T327 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 107123 T327 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 107124 T327 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 107124 T327 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 107125 T327 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 107125 T327 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 107126 T327 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from instancedir /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/
   [junit4]   2> 107126 T327 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 107127 T327 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.component.DistributedFacetPivotSmallTest-80F928B7C834E5E4-001/tempDir-001/shard3/data/
   [junit4]   2> 107127 T327 oasc.JmxMonitoredMap.<init> No JMX servers found, not exposing Solr information with JMX.
   [junit4]   2> 107128 T327 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.component.DistributedFacetPivotSmallTest-80F928B7C834E5E4-001/tempDir-001/shard3/data
   [junit4]   2> 107128 T327 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.component.DistributedFacetPivotSmallTest-80F928B7C834E5E4-001/tempDir-001/shard3/data/index/
   [junit4]   2> 107128 T327 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.component.DistributedFacetPivotSmallTest-80F928B7C834E5E4-001/tempDir-001/shard3/data/index' doesn't exist. Creating new index...
   [junit4]   2> 107129 T327 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.handler.component.DistributedFacetPivotSmallTest-80F928B7C834E5E4-001/tempDir-001/shard3/data/index
   [junit4]   2> 107129 T327 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=17, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 107131 T327 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@700a936b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@46548f44; maxCacheMB=0.5673828125 maxMergeSizeMB=0.9296875))),segFN=segments_1,generation=1}
   [junit4]   2> 107131 T327 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 107131 T327 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
   [junit4]   2> 107133 T327 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 107134 T327 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 107134 T327 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 107134 T327 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 107134 T327 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 107135 T327 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 107135 T327 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 107136 T327 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 107136 T327 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 107136 T327 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 107137 T327 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 107137 T327 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 107138 T327 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 107138 T327 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 107139 T327 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 107139 T327 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 107139 T327 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 107140 T327 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 107140 T327 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 107141 T327 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
   [junit4]   2> 107141 T327 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
   [junit4]   2> 107141 T327 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 107142 T327 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 107142 T327 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 107143 T327 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
   [junit4]   2> 107143 T327 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
   [junit4]   2> 107143 T327 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
   [junit4]   2> 107144 T327 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 107144 T327 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 107144 T327 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 107145 T327 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 107145 T327 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 107145 T327 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 107146 T327 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 107146 T327 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 107146 T327 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 107147 T327 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
   [junit4]   2> 107147 T327 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
   [junit4]   2> 107169 T327 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 107172 T327 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 107175 T327 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 107177 T327 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 107179 T327 oasc.SolrCore.initDeprecatedSupport WARN solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
   [junit4]   2> 107181 T327 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 107181 T327 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 107182 T327 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=35, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 107183 T327 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@700a936b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@46548f44; maxCacheMB=0.5673828125 maxMergeSizeMB=0.9296875))),segFN=segments_1,generation=1}
   [junit4]   2> 107183 T327 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 107184 T327 oass.SolrIndexSearcher.<init> Opening Searcher@68ab1c2d[collection1] main
   [junit4]   2> 107184 T327 oasr.RestManager.init Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
   [junit4]   2> 107185 T327 oasr.ManagedResourceStorage.load Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 107185 T327 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /rest/managed
   [junit4]   2> 107185 T327 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using InMemoryStorage
   [junit4]   2> 107186 T327 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 107186 T327 oashc.SpellCheckComponent.inform Initializing spell checkers
   [junit4]   2> 107229 T327 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 107295 T328 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@68ab1c2d[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 107295 T327 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 107296 T269 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 107297 T269 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> ASYNC  NEW_CORE C26 name=collection1 org.apache.solr.core.SolrCore@72e8de2e
   [junit4]   2> 107308 T272 C26 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 107309 T272 C26 oasup.LogUpdateProcessor.finish [collection1] webapp=/_o/ub path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 2
   [junit4]   2> ASYNC  NEW_CORE C27 name=collection1 org.apache.solr.core.SolrCore@59909ac7
   [junit4]   2> 107315 T284 C27 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 107316 T284 C27 oasup.LogUpdateProcessor.finish [collection1] webapp=/_o/ub path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 1
   [junit4]   2> ASYNC  NEW_CORE C28 name=collection1 org.apache.solr.core.SolrCore@3f9c4b9a
   [junit4]   2> 107334 T296 C28 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 107335 T296 C28 oasup.LogUpdateProcessor.finish [collection1] webapp=/_o/ub path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 2
   [junit4]   2> ASYNC  NEW_CORE C29 name=collection1 org.apache.solr.core.SolrCore@8cc15ac
   [junit4]   2> 107339 T308 C29 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 107339 T308 C29 oasup.LogUpdateProcessor.finish [collection1] webapp=/_o/ub path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 1
   [junit4]   2> ASYNC  NEW_CORE C30 name=collection1 org.apache.solr.core.SolrCore@501d19a
   [junit4]   2> 107360 T320 C30 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 107361 T320 C30 oasup.LogUpdateProcessor.finish [collection1] webapp=/_o/ub path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 1
   [junit4]   2> 107369 T272 C26 oasup.LogUpdateProcessor.finish [collection1] webapp=/_o/ub path=/update params={wt=javabin&version=2} {add=[19]} 0 4
   [junit4]   2> 107375 T308 C29 oasup.LogUpdateProcessor.finish [collection1] webapp=/_o/ub path=/update params={wt=javabin&version=2} {add=[19]} 0 3
   [junit4]   2> 107383 T272 C26 oasup.LogUpdateProcessor.finish [collection1] webapp=/_o/ub path=/update params={wt=javabin&version=2} {add=[20]} 0 3
   [junit4]   2> 107395 T284 C27 oasup.LogUpdateProcessor.finish [collection1] webapp=/_o/ub path=/update params={wt=javabin&version=2} {add=[20]} 0 6
   [junit4]   2> 107401 T272 C26 oasup.LogUpdateProcessor.finish [collection1] webapp=/_o/ub path=/update params={wt=javabin&version=2} {add=[21]} 0 2
   [junit4]   2> 107409 T296 C28 oasup.LogUpdateProcessor.finish [collection1] webapp=/_o/ub path=/update params={wt=javabin&version=2} {add=[21]} 0 4
   [junit4]   2> 107414 T272 C26 oasup.LogUpdateProcessor.finish [collection1] webapp=/_o/ub path=/update params={wt=javabin&version=2} {add=[22]} 0 2
   [junit4]   2> 107419 T308 C29 oasup.LogUpdateProcessor.finish [collection1] webapp=/_o/ub path=/update params={wt=javabin&version=2} {add=[22]} 0 2
   [junit4]   2> 107424 T272 C26 oasup.LogUpdateProcessor.finish [collection1] webapp=/_o/ub path=/update params={wt=javabin&version=2} {add=[23]} 0 2
   [junit4]   2> 107432 T320 C30 oasup.LogUpdateProcessor.finish [collection1] webapp=/_o/ub path=/update params={wt=javabin&version=2} {add=[23]} 0 5
   [junit4]   2> 107438 T272 C26 oasup.LogUpdateProcessor.finish [collection1] webapp=/_o/ub path=/update params={wt=javabin&version=2} {add=[24]} 0 2
   [junit4]   2> 107442 T284 C27 oasup.LogUpdateProcessor.finish [collection1] webapp=/_o/ub path=/update params={wt=javabin&version=2} {add=[24]} 0 1
   [junit4]   2> 107447 T272 C26 oasup.LogUpdateProcessor.finish [collection1] webapp=/_o/ub path=/update params={wt=javabin&version=2} {add=[25]} 0 1
   [junit4]   2> 107453 T296 C28 oasup.LogUpdateProcessor.finish [collection1] webapp=/_o/ub path=/update params={wt=javabin&version=2} {add=[25]} 0 2
   [junit4]   2> 107458 T272 C26 oasup.LogUpdateProcessor.finish [collection1] webapp=/_o/ub path=/update pa

[...truncated too long message...]

rkQueue.take(ScheduledThreadPoolExecutor.java:1079)
   [junit4]   2> 	        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	  27) Thread[id=3854, name=Thread-1200-SendThread(127.0.0.1:15930), state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
   [junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:85)
   [junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
   [junit4]   2> 	        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
   [junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
   [junit4]   2> 	  28) Thread[id=3540, name=Thread-1162, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
   [junit4]   2> 	        at java.lang.Object.wait(Native Method)
   [junit4]   2> 	        at java.lang.Object.wait(Object.java:503)
   [junit4]   2> 	        at org.apache.solr.core.CloserThread.run(CoreContainer.java:923)
   [junit4]   2> 	  29) Thread[id=3634, name=commitScheduler-1313-thread-1, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2> 	        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
   [junit4]   2> 	        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	  30) Thread[id=3524, name=qtp1781045729-3524 Acceptor0 SslSocketConnector@127.0.0.1:16067, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
   [junit4]   2> 	        at java.net.PlainSocketImpl.socketAccept(Native Method)
   [junit4]   2> 	        at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
   [junit4]   2> 	        at java.net.ServerSocket.implAccept(ServerSocket.java:530)
   [junit4]   2> 	        at sun.security.ssl.SSLServerSocketImpl.accept(SSLServerSocketImpl.java:317)
   [junit4]   2> 	        at org.eclipse.jetty.server.ssl.SslSocketConnector.accept(SslSocketConnector.java:108)
   [junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	  31) Thread[id=3504, name=qtp765537004-3504 Acceptor0 SslSocketConnector@127.0.0.1:15994, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
   [junit4]   2> 	        at java.net.PlainSocketImpl.socketAccept(Native Method)
   [junit4]   2> 	        at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
   [junit4]   2> 	        at java.net.ServerSocket.implAccept(ServerSocket.java:530)
   [junit4]   2> 	        at sun.security.ssl.SSLServerSocketImpl.accept(SSLServerSocketImpl.java:317)
   [junit4]   2> 	        at org.eclipse.jetty.server.ssl.SslSocketConnector.accept(SslSocketConnector.java:108)
   [junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	  32) Thread[id=3843, name=qtp1303024931-3843 Acceptor0 SslSocketConnector@127.0.0.1:16082, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
   [junit4]   2> 	        at java.net.PlainSocketImpl.socketAccept(Native Method)
   [junit4]   2> 	        at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
   [junit4]   2> 	        at java.net.ServerSocket.implAccept(ServerSocket.java:530)
   [junit4]   2> 	        at sun.security.ssl.SSLServerSocketImpl.accept(SSLServerSocketImpl.java:317)
   [junit4]   2> 	        at org.eclipse.jetty.server.ssl.SslSocketConnector.accept(SslSocketConnector.java:108)
   [junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene50): {id=PostingsFormat(name=Asserting), a_i=PostingsFormat(name=Asserting), _version_=PostingsFormat(name=Direct), a_t=BlockTreeOrds(blocksize=128), rnd_b=PostingsFormat(name=Asserting)}, docValues:{}, sim=DefaultSimilarity, locale=ca_ES, timezone=America/Barbados
   [junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_65 (64-bit)/cpus=16,threads=34,free=49383200,total=259522560
   [junit4]   2> NOTE: All tests run in this JVM: [QueryEqualityTest, FileUtilsTest, CacheHeaderTest, TestBulkSchemaConcurrent, DocValuesMissingTest, HdfsDirectoryTest, DistributedQueryElevationComponentTest, TestRandomMergePolicy, SolrXmlInZkTest, DirectSolrConnectionTest, TestDefaultSimilarityFactory, TestValueSourceCache, SimplePostToolTest, TestWriterPerf, TestXIncludeConfig, TestArbitraryIndexDir, BlockCacheTest, TestStandardQParsers, TestSolrDeletionPolicy1, JsonLoaderTest, ResourceLoaderTest, IndexSchemaRuntimeFieldTest, TestQueryTypes, SchemaVersionSpecificBehaviorTest, TestSimpleQParserPlugin, TestSolrJ, TestMiniSolrCloudCluster, SimpleCollectionCreateDeleteTest, ConnectionManagerTest, TestImplicitCoreProperties, TestNRTOpen, TestDynamicFieldCollectionResource, PrimUtilsTest, HdfsLockFactoryTest, HdfsCollectionsAPIDistributedZkTest, TestNonNRTOpen, TestRestManager, TestCustomSort, SuggesterWFSTTest, TestDistribDocBasedVersion, TestAnalyzedSuggestions, TestStressReorder, TestLazyCores, SolrCmdDistributorTest, OverseerTest, TestRecoveryHdfs, ConvertedLegacyTest, SynonymTokenizerTest, DistributedFacetPivotSmallAdvancedTest, ParsingFieldUpdateProcessorsTest, TestLeaderElectionZkExpiry, TestRangeQuery, TestBinaryResponseWriter, ScriptEngineTest, MoreLikeThisHandlerTest, DistributedSpellCheckComponentTest, BasicDistributedZkTest, ChaosMonkeySafeLeaderTest]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ChaosMonkeySafeLeaderTest -Dtests.seed=80F928B7C834E5E4 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=ca_ES -Dtests.timezone=America/Barbados -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J0 | ChaosMonkeySafeLeaderTest (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([80F928B7C834E5E4]:0)
   [junit4] Completed on J0 in 7224.02s, 1 test, 2 errors <<< FAILURES!

[...truncated 14 lines...]
BUILD FAILED
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:532: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:473: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:61: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/extra-targets.xml:39: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:508: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1363: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:970: There were test failures: 445 suites, 1819 tests, 1 suite-level error, 2 errors, 54 ignored (9 assumptions)

Total time: 277 minutes 16 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #680
Archived 5 artifacts
Archive block size is 32768
Received 0 blocks and 1158146612 bytes
Compression is 0.0%
Took 3 min 30 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure