You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2017/07/09 05:01:37 UTC
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 5 - Still Unstable
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/5/
2 tests failed.
FAILED: org.apache.lucene.codecs.memory.TestMemoryDocValuesFormat.testSparseSortedFixedLengthVsStoredFields
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([623D4CC06F663EA4]:0)
FAILED: junit.framework.TestSuite.org.apache.lucene.codecs.memory.TestMemoryDocValuesFormat
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([623D4CC06F663EA4]:0)
Build Log:
[...truncated 6731 lines...]
[junit4] Suite: org.apache.lucene.codecs.memory.TestMemoryDocValuesFormat
[junit4] 2> 7월 09, 2017 4:19:29 오전 com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
[junit4] 2> WARNING: Suite execution timed out: org.apache.lucene.codecs.memory.TestMemoryDocValuesFormat
[junit4] 2> 1) Thread[id=11, name=SUITE-TestMemoryDocValuesFormat-seed#[623D4CC06F663EA4], state=RUNNABLE, group=TGRP-TestMemoryDocValuesFormat]
[junit4] 2> at java.lang.Thread.getStackTrace(Thread.java:1559)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:696)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:693)
[junit4] 2> at java.security.AccessController.doPrivileged(Native Method)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:693)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:709)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:689)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:65)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:415)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:678)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:140)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:598)
[junit4] 2> 2) Thread[id=9, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:50)
[junit4] 2> 3) Thread[id=1, name=main, state=WAITING, group=main]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Thread.join(Thread.java:1252)
[junit4] 2> at java.lang.Thread.join(Thread.java:1326)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:608)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:457)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:251)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:368)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:13)
[junit4] 2> 4) Thread[id=12, name=TEST-TestMemoryDocValuesFormat.testSparseSortedFixedLengthVsStoredFields-seed#[623D4CC06F663EA4], state=TIMED_WAITING, group=TGRP-TestMemoryDocValuesFormat]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at org.apache.lucene.index.IndexWriter.doWait(IndexWriter.java:4628)
[junit4] 2> at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:2091)
[junit4] 2> at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:2019)
[junit4] 2> at org.apache.lucene.index.RandomIndexWriter.forceMerge(RandomIndexWriter.java:429)
[junit4] 2> at org.apache.lucene.index.BaseDocValuesFormatTestCase.doTestSortedVsStoredFields(BaseDocValuesFormatTestCase.java:1591)
[junit4] 2> at org.apache.lucene.index.BaseDocValuesFormatTestCase.doTestSortedVsStoredFields(BaseDocValuesFormatTestCase.java:1648)
[junit4] 2> at org.apache.lucene.index.BaseDocValuesFormatTestCase.testSparseSortedFixedLengthVsStoredFields(BaseDocValuesFormatTestCase.java:1629)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestMemoryDocValuesFormat -Dtests.method=testSparseSortedFixedLengthVsStoredFields -Dtests.seed=623D4CC06F663EA4 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=ko-KR -Dtests.timezone=Egypt -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 7014s J0 | TestMemoryDocValuesFormat.testSparseSortedFixedLengthVsStoredFields <<<
[junit4] > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
[junit4] > at __randomizedtesting.SeedInfo.seed([623D4CC06F663EA4]:0)
[junit4] 2> 7월 09, 2017 4:19:29 오전 com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> 7월 09, 2017 4:19:49 오전 com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.codecs.memory.TestMemoryDocValuesFormat:
[junit4] 2> 1) Thread[id=12, name=TEST-TestMemoryDocValuesFormat.testSparseSortedFixedLengthVsStoredFields-seed#[623D4CC06F663EA4], state=TIMED_WAITING, group=TGRP-TestMemoryDocValuesFormat]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at org.apache.lucene.index.IndexWriter.doWait(IndexWriter.java:4628)
[junit4] 2> at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:2091)
[junit4] 2> at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:2019)
[junit4] 2> at org.apache.lucene.index.RandomIndexWriter.forceMerge(RandomIndexWriter.java:429)
[junit4] 2> at org.apache.lucene.index.BaseDocValuesFormatTestCase.doTestSortedVsStoredFields(BaseDocValuesFormatTestCase.java:1591)
[junit4] 2> at org.apache.lucene.index.BaseDocValuesFormatTestCase.doTestSortedVsStoredFields(BaseDocValuesFormatTestCase.java:1648)
[junit4] 2> at org.apache.lucene.index.BaseDocValuesFormatTestCase.testSparseSortedFixedLengthVsStoredFields(BaseDocValuesFormatTestCase.java:1629)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 7월 09, 2017 4:19:49 오전 com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=12, name=TEST-TestMemoryDocValuesFormat.testSparseSortedFixedLengthVsStoredFields-seed#[623D4CC06F663EA4], state=TIMED_WAITING, group=TGRP-TestMemoryDocValuesFormat]
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/build/codecs/test/J0/temp/lucene.codecs.memory.TestMemoryDocValuesFormat_623D4CC06F663EA4-001
[junit4] 2> Jul 09, 2017 2:19:49 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: All leaked threads terminated.
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=800, maxMBSortInHeap=6.291068436637745, sim=RandomSimilarity(queryNorm=false): {fieldname=DFR I(n)B2}, locale=ko-KR, timezone=Egypt
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_131 (64-bit)/cpus=4,threads=1,free=213825664,total=319291392
[junit4] 2> NOTE: All tests run in this JVM: [TestMemoryDocValuesFormat]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestMemoryDocValuesFormat -Dtests.seed=623D4CC06F663EA4 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=ko-KR -Dtests.timezone=Egypt -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J0 | TestMemoryDocValuesFormat (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([623D4CC06F663EA4]:0)
[junit4] Completed [18/20 (1!)] on J0 in 7220.46s, 59 tests, 2 errors <<< FAILURES!
[...truncated 50330 lines...]
Re: [JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 10 - Still
unstable
Posted by Alan Woodward <al...@romseysoftware.co.uk>.
I pushed a fix - SerializedDVStrategy was calling reader.getBinaryDocValues() rather than DocValues.getBinary(), and erroring out if it hit a segment with no binary values indexed.
> On 15 Jul 2017, at 20:11, David Smiley <da...@gmail.com> wrote:
>
> Alan, I suspect this is related the changes you did?
>
> On Sat, Jul 15, 2017 at 6:07 AM Apache Jenkins Server <jenkins@builds.apache.org <ma...@builds.apache.org>> wrote:
> Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/10/ <https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/10/>
>
> 3 tests failed.
> FAILED: org.apache.lucene.spatial.DistanceStrategyTest.testRecipScore {strategy=serialized}
>
> Error Message:
>
>
> Stack Trace:
> java.lang.NullPointerException
> at __randomizedtesting.SeedInfo.seed([B1FC2FB143140AF1:29D25F48109F000A]:0)
> at org.apache.lucene.spatial.serialized.SerializedDVStrategy$ShapeDocValueSource$1.advanceExact(SerializedDVStrategy.java:186)
> at org.apache.lucene.spatial.util.DistanceToShapeValueSource$1.advanceExact(DistanceToShapeValueSource.java:73)
> at org.apache.lucene.search.DoubleValues$1.advanceExact(DoubleValues.java:53)
> at org.apache.lucene.spatial.util.ReciprocalDoubleValuesSource$1.advanceExact(ReciprocalDoubleValuesSource.java:57)
> at org.apache.lucene.spatial.StrategyTestCase.checkValueSource(StrategyTestCase.java:219)
> at org.apache.lucene.spatial.DistanceStrategyTest.testRecipScore(DistanceStrategyTest.java:104)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at java.lang.Thread.run(Thread.java:748)
>
>
> FAILED: org.apache.solr.update.AutoCommitTest.testMaxDocs
>
> Error Message:
> Exception during query
>
> Stack Trace:
> java.lang.RuntimeException: Exception during query
> at __randomizedtesting.SeedInfo.seed([793723D4943B9E3D:C0B6F50BB8D19AB7]:0)
> at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:878)
> at org.apache.solr.update.AutoCommitTest.testMaxDocs(AutoCommitTest.java:225)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
> xml response was: <?xml version="1.0" encoding="UTF-8"?>
> <response>
> <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
> </response>
>
> request was:q=id:14&qt=&start=0&rows=20&version=2.2
> at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:871)
> ... 40 more
>
>
> FAILED: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test
>
> Error Message:
> Timeout waiting for all live and active
>
> Stack Trace:
> java.lang.AssertionError: Timeout waiting for all live and active
> at __randomizedtesting.SeedInfo.seed([793723D4943B9E3D:F1631C0E3AC7F3C5]:0)
> at org.junit.Assert.fail(Assert.java:93)
> at org.junit.Assert.assertTrue(Assert.java:43)
> at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:246)
> at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:136)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
> at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
> at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at java.lang.Thread.run(Thread.java:748)
>
>
>
>
> Build Log:
> [...truncated 9261 lines...]
> [junit4] Suite: org.apache.lucene.spatial.DistanceStrategyTest
> [junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
> [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=DistanceStrategyTest -Dtests.method=testRecipScore -Dtests.seed=B1FC2FB143140AF1 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=ar-AE -Dtests.timezone=Pacific/Chatham -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
> [junit4] ERROR 0.03s J0 | DistanceStrategyTest.testRecipScore {strategy=serialized} <<<
> [junit4] > Throwable #1: java.lang.NullPointerException
> [junit4] > at __randomizedtesting.SeedInfo.seed([B1FC2FB143140AF1:29D25F48109F000A]:0)
> [junit4] > at org.apache.lucene.spatial.serialized.SerializedDVStrategy$ShapeDocValueSource$1.advanceExact(SerializedDVStrategy.java:186)
> [junit4] > at org.apache.lucene.spatial.util.DistanceToShapeValueSource$1.advanceExact(DistanceToShapeValueSource.java:73)
> [junit4] > at org.apache.lucene.search.DoubleValues$1.advanceExact(DoubleValues.java:53)
> [junit4] > at org.apache.lucene.spatial.util.ReciprocalDoubleValuesSource$1.advanceExact(ReciprocalDoubleValuesSource.java:57)
> [junit4] > at org.apache.lucene.spatial.StrategyTestCase.checkValueSource(StrategyTestCase.java:219)
> [junit4] > at org.apache.lucene.spatial.DistanceStrategyTest.testRecipScore(DistanceStrategyTest.java:104)
> [junit4] > at java.lang.Thread.run(Thread.java:748)
> [junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {recursive_packedquad=PostingsFormat(name=MockRandom), recursive_quad=PostingsFormat(name=MockRandom), bbox__xdl=BlockTreeOrds(blocksize=128), termquery_geohash=PostingsFormat(name=MockRandom), id=BlockTreeOrds(blocksize=128)}, docValues:{bbox__maxX=DocValuesFormat(name=Direct), bbox__maxY=DocValuesFormat(name=Lucene70), bbox__minX=DocValuesFormat(name=Asserting), bbox__minY=DocValuesFormat(name=Memory), pointvector__y=DocValuesFormat(name=Lucene70), pointvector__x=DocValuesFormat(name=Direct), serialized=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=1913, maxMBSortInHeap=5.545002168373545, sim=RandomSimilarity(queryNorm=true): {}, locale=ar-AE, timezone=Pacific/Chatham
> [junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_131 (64-bit)/cpus=4,threads=1,free=291830768,total=324534272
> [junit4] 2> NOTE: All tests run in this JVM: [SpatialPrefixTreeTest, NumberRangeFacetsTest, TestTermQueryPrefixGridStrategy, TestRecursivePrefixTreeStrategy, Geo3dShapeSphereModelRectRelationTest, DistanceStrategyTest]
> [junit4] Completed [12/23 (1!)] on J0 in 0.69s, 12 tests, 1 error <<< FAILURES!
>
> [...truncated 2297 lines...]
> [junit4] Suite: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest
> [junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/init-core-data-001
> [junit4] 2> 548632 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=38 numCloses=38
> [junit4] 2> 548632 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
> [junit4] 2> 548634 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
> [junit4] 2> 548634 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /ov/cn
> [junit4] 1> Formatting using clusterid: testClusterID
> [junit4] 2> 548849 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
> [junit4] 2> 548855 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
> [junit4] 2> 548857 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.m.log jetty-6.1.26
> [junit4] 2> 548870 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_56321_hdfs____.206o6d/webapp
> [junit4] 2> 549299 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:56321
> [junit4] 2> 549422 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
> [junit4] 2> 549423 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.m.log jetty-6.1.26
> [junit4] 2> 549436 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_59355_datanode____.vqlzu2/webapp
> [junit4] 2> 549878 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:59355
> [junit4] 2> 549915 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
> [junit4] 2> 549916 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.m.log jetty-6.1.26
> [junit4] 2> 549934 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_54759_datanode____jrk8hv/webapp
> [junit4] 2> 550164 INFO (IPC Server handler 4 on 49971) [ ] BlockStateChange BLOCK* processReport: from storage DS-e9ea3dbc-9113-4645-b893-130c3730ec1a node DatanodeRegistration(127.0.0.1:50603 <http://127.0.0.1:50603/>, datanodeUuid=527ac2ae-b401-4d84-86e3-6f935dee3843, infoPort=54653, infoSecurePort=0, ipcPort=36188, storageInfo=lv=-56;cid=testClusterID;nsid=451575161;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
> [junit4] 2> 550164 INFO (IPC Server handler 4 on 49971) [ ] BlockStateChange BLOCK* processReport: from storage DS-ada032ef-beb3-4321-b044-13278d645633 node DatanodeRegistration(127.0.0.1:50603 <http://127.0.0.1:50603/>, datanodeUuid=527ac2ae-b401-4d84-86e3-6f935dee3843, infoPort=54653, infoSecurePort=0, ipcPort=36188, storageInfo=lv=-56;cid=testClusterID;nsid=451575161;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
> [junit4] 2> 550420 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:54759
> [junit4] 2> 550918 INFO (IPC Server handler 9 on 49971) [ ] BlockStateChange BLOCK* processReport: from storage DS-7b441025-265d-40cb-ac06-5e2d91c89e3b node DatanodeRegistration(127.0.0.1:60224 <http://127.0.0.1:60224/>, datanodeUuid=48e239a8-7fb1-4b7a-bad7-3965b9821989, infoPort=45761, infoSecurePort=0, ipcPort=36307, storageInfo=lv=-56;cid=testClusterID;nsid=451575161;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
> [junit4] 2> 550918 INFO (IPC Server handler 9 on 49971) [ ] BlockStateChange BLOCK* processReport: from storage DS-4f0f9719-f907-44eb-a680-fe795758da6b node DatanodeRegistration(127.0.0.1:60224 <http://127.0.0.1:60224/>, datanodeUuid=48e239a8-7fb1-4b7a-bad7-3965b9821989, infoPort=45761, infoSecurePort=0, ipcPort=36307, storageInfo=lv=-56;cid=testClusterID;nsid=451575161;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
> [junit4] 2> 551053 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
> [junit4] 2> 551060 INFO (Thread-2097) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0 <http://0.0.0.0/0.0.0.0:0>
> [junit4] 2> 551060 INFO (Thread-2097) [ ] o.a.s.c.ZkTestServer Starting server
> [junit4] 2> 551075 ERROR (Thread-2097) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
> [junit4] 2> 551160 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.ZkTestServer start zk server on port:52694
> [junit4] 2> 551179 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
> [junit4] 2> 551181 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
> [junit4] 2> 551183 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
> [junit4] 2> 551185 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
> [junit4] 2> 551187 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
> [junit4] 2> 551189 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
> [junit4] 2> 551191 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
> [junit4] 2> 551192 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
> [junit4] 2> 551194 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
> [junit4] 2> 551196 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
> [junit4] 2> 551198 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
> [junit4] 2> 551201 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
> [junit4] 2> 551336 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
> [junit4] 2> 551338 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@16cae510{/ov/cn,null,AVAILABLE}
> [junit4] 2> 551338 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@597fe4f4{HTTP/1.1,[http/1.1]}{127.0.0.1:41805 <http://127.0.0.1:41805/>}
> [junit4] 2> 551338 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.e.j.s.Server Started @554231ms
> [junit4] 2> 551338 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/ov/cn, hostPort=41805, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/control-001/cores}
> [junit4] 2> 551338 ERROR (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
> [junit4] 2> 551340 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
> [junit4] 2> 551340 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
> [junit4] 2> 551340 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
> [junit4] 2> 551340 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-07-15T08:38:42.901Z
> [junit4] 2> 551346 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
> [junit4] 2> 551346 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/control-001/solr.xml
> [junit4] 2> 551352 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de, but no JMX reporters were configured - adding default JMX reporter.
> [junit4] 2> 551355 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52694/solr <http://127.0.0.1:52694/solr>
> [junit4] 2> 551468 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.Overseer Overseer (id=null) closing
> [junit4] 2> 551469 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:41805_ov%2Fcn
> [junit4] 2> 551470 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.Overseer Overseer (id=98311072817348612-127.0.0.1:41805_ov%2Fcn-n_0000000000) starting
> [junit4] 2> 551480 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0. <http://127.0.0.0/>1:41805_ov%2Fcn
> [junit4] 2> 551482 INFO (zkCallback-337-thread-1-processing-n:127.0.0.1:41805_ov%2Fcn) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
> [junit4] 2> 551605 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
> [junit4] 2> 551613 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
> [junit4] 2> 551613 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
> [junit4] 2> 551615 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/control-001/cores
> [junit4] 2> 551639 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
> [junit4] 2> 551640 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:52694/solr <http://127.0.0.1:52694/solr> ready
> [junit4] 2> 551642 INFO (qtp1213643546-4367) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:41805_ov%252Fcn&wt=javabin&version=2 and sendToOCPQueue=true
> [junit4] 2> 551645 INFO (OverseerThreadFactory-758-thread-1-processing-n:127.0.0.1:41805_ov%2Fcn) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.CreateCollectionCmd Create collection control_collection
> [junit4] 2> 551750 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
> [junit4] 2> 551751 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
> [junit4] 2> 551858 INFO (zkCallback-337-thread-1-processing-n:127.0.0.1:41805_ov%2Fcn) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
> [junit4] 2> 552773 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
> [junit4] 2> 552787 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
> [junit4] 2> 552875 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
> [junit4] 2> 552895 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
> [junit4] 2> 552895 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
> [junit4] 2> 552895 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:49971/solr_hdfs_home
> [junit4] 2> 552895 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
> [junit4] 2> 552895 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
> [junit4] 2> 552896 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[hdfs://localhost:49971/solr_hdfs_home/control_collection/core_node1/data/]
> [junit4] 2> 552907 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49971/solr_hdfs_home/control_collection/core_node1/data/snapshot_metadata
> [junit4] 2> 552916 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
> [junit4] 2> 552916 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
> [junit4] 2> 552917 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Creating new global HDFS BlockCache
> [junit4] 2> 552925 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
> [junit4] 2> 552926 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49971/solr_hdfs_home/control_collection/core_node1/data
> [junit4] 2> 552951 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49971/solr_hdfs_home/control_collection/core_node1/data/index
> [junit4] 2> 552958 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
> [junit4] 2> 552958 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
> [junit4] 2> 552963 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
> [junit4] 2> 552964 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=7, maxMergeAtOnceExplicit=2, maxMergedSegmentMB=54.2822265625, floorSegmentMB=1.00390625, forceMergeDeletesPctAllowed=4.833164813857602, segmentsPerTier=10.0, maxCFSSegmentSizeMB=0.6904296875, noCFSRatio=0.0
> [junit4] 2> 553000 INFO (IPC Server handler 9 on 49971) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:60224 <http://127.0.0.1:60224/> is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-e9ea3dbc-9113-4645-b893-130c3730ec1a:NORMAL:127.0.0.1 <http://127.0.0.1/>:50603|RBW], ReplicaUC[[DISK]DS-7b441025-265d-40cb-ac06-5e2d91c89e3b:NORMAL:127.0.0.1:60224|FINALIZED]]} size 0
> [junit4] 2> 553007 INFO (IPC Server handler 3 on 49971) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:50603 <http://127.0.0.1:50603/> is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-7b441025-265d-40cb-ac06-5e2d91c89e3b:NORMAL:127.0.0.1 <http://127.0.0.1/>:60224|FINALIZED], ReplicaUC[[DISK]DS-ada032ef-beb3-4321-b044-13278d645633:NORMAL:127.0.0.1:50603|FINALIZED]]} size 0
> [junit4] 2> 553015 WARN (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
> [junit4] 2> 553071 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
> [junit4] 2> 553071 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
> [junit4] 2> 553071 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
> [junit4] 2> 553096 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
> [junit4] 2> 553096 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
> [junit4] 2> 553103 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=13, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=10.5625, floorSegmentMB=0.54296875, forceMergeDeletesPctAllowed=17.113764658263925, segmentsPerTier=33.0, maxCFSSegmentSizeMB=2.0732421875, noCFSRatio=1.0
> [junit4] 2> 553112 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@144f5f33[control_collection_shard1_replica_n1] main]
> [junit4] 2> 553123 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
> [junit4] 2> 553124 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
> [junit4] 2> 553125 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
> [junit4] 2> 553127 INFO (searcherExecutor-761-thread-1-processing-n:127.0.0.1:41805 <http://127.0.0.1:41805/>_ov%2Fcn x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@144f5f33[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
> [junit4] 2> 553136 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1572977167247081472
> [junit4] 2> 553144 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
> [junit4] 2> 553144 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
> [junit4] 2> 553144 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:41805/ov/cn/control_collection_shard1_replica_n1/ <http://127.0.0.1:41805/ov/cn/control_collection_shard1_replica_n1/>
> [junit4] 2> 553144 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
> [junit4] 2> 553144 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:41805/ov/cn/control_collection_shard1_replica_n1/ <http://127.0.0.1:41805/ov/cn/control_collection_shard1_replica_n1/> has no replicas
> [junit4] 2> 553144 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
> [junit4] 2> 553172 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:41805/ov/cn/control_collection_shard1_replica_n1/ <http://127.0.0.1:41805/ov/cn/control_collection_shard1_replica_n1/> shard1
> [junit4] 2> 553274 INFO (zkCallback-337-thread-1-processing-n:127.0.0.1:41805_ov%2Fcn) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
> [junit4] 2> 553324 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
> [junit4] 2> 553326 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1575
> [junit4] 2> 553333 INFO (qtp1213643546-4367) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
> [junit4] 2> 553435 INFO (zkCallback-337-thread-1-processing-n:127.0.0.1:41805_ov%2Fcn) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
> [junit4] 2> 553649 INFO (OverseerCollectionConfigSetProcessor-98311072817348612-127.0.0.1:41805_ov%2Fcn-n_0000000000) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
> [junit4] 2> 554333 INFO (qtp1213643546-4367) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:41805_ov%252Fcn&wt=javabin&version=2} status=0 QTime=2691
> [junit4] 2> 554341 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
> [junit4] 2> 554342 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:52694/solr <http://127.0.0.1:52694/solr> ready
> [junit4] 2> 554342 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
> [junit4] 2> 554343 INFO (qtp1213643546-4368) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
> [junit4] 2> 554345 INFO (OverseerThreadFactory-758-thread-2-processing-n:127.0.0.1:41805_ov%2Fcn) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.CreateCollectionCmd Create collection collection1
> [junit4] 2> 554347 WARN (OverseerThreadFactory-758-thread-2-processing-n:127.0.0.1:41805_ov%2Fcn) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
> [junit4] 2> 554558 INFO (qtp1213643546-4368) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
> [junit4] 2> 554558 INFO (qtp1213643546-4368) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2} status=0 QTime=214
> [junit4] 2> 554704 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/shard-1-001 of type NRT
> [junit4] 2> 554705 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
> [junit4] 2> 554707 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1ee5d6f8{/ov/cn,null,AVAILABLE}
> [junit4] 2> 554708 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2bdfb805{HTTP/1.1,[http/1.1]}{127.0.0.1:32870 <http://127.0.0.1:32870/>}
> [junit4] 2> 554708 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.e.j.s.Server Started @557601ms
> [junit4] 2> 554708 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, hostContext=/ov/cn, hostPort=32870, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/shard-1-001/cores}
> [junit4] 2> 554708 ERROR (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
> [junit4] 2> 554708 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
> [junit4] 2> 554708 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
> [junit4] 2> 554708 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
> [junit4] 2> 554708 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-07-15T08:38:46.269Z
> [junit4] 2> 554722 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
> [junit4] 2> 554722 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/shard-1-001/solr.xml
> [junit4] 2> 554729 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de, but no JMX reporters were configured - adding default JMX reporter.
> [junit4] 2> 554732 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52694/solr <http://127.0.0.1:52694/solr>
> [junit4] 2> 554746 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
> [junit4] 2> 554749 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.c.Overseer Overseer (id=null) closing
> [junit4] 2> 554752 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0. <http://127.0.0.0/>1:32870_ov%2Fcn
> [junit4] 2> 554753 INFO (zkCallback-344-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
> [junit4] 2> 554755 INFO (zkCallback-337-thread-1-processing-n:127.0.0.1:41805_ov%2Fcn) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
> [junit4] 2> 554760 INFO (zkCallback-349-thread-1-processing-n:127.0.0.1:32870_ov%2Fcn) [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
> [junit4] 2> 554850 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
> [junit4] 2> 554856 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
> [junit4] 2> 554856 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
> [junit4] 2> 554857 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/shard-1-001/cores
> [junit4] 2> 554887 INFO (qtp379993423-4422) [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:32870_ov%252Fcn&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
> [junit4] 2> 554890 INFO (OverseerThreadFactory-758-thread-3-processing-n:127.0.0.1:41805_ov%2Fcn) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:32870_ov%2Fcn for creating new replica
> [junit4] 2> 554890 INFO (OverseerCollectionConfigSetProcessor-98311072817348612-127.0.0.1:41805_ov%2Fcn-n_0000000000) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
> [junit4] 2> 554893 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n0&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
> [junit4] 2> 554893 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
> [junit4] 2> 555915 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
> [junit4] 2> 555932 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.s.IndexSchema [collection1_shard2_replica_n0] Schema name=test
> [junit4] 2> 556029 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
> [junit4] 2> 556051 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n0' using configuration from collection collection1, trusted=true
> [junit4] 2> 556052 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n0' (registry 'solr.core.collection1.shard2.replica_n0') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
> [junit4] 2> 556052 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:49971/solr_hdfs_home
> [junit4] 2> 556052 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
> [junit4] 2> 556052 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
> [junit4] 2> 556052 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.SolrCore [[collection1_shard2_replica_n0] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/shard-1-001/cores/collection1_shard2_replica_n0], dataDir=[hdfs://localhost:49971/solr_hdfs_home/collection1/core_node1/data/]
> [junit4] 2> 556054 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49971/solr_hdfs_home/collection1/core_node1/data/snapshot_metadata
> [junit4] 2> 556064 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
> [junit4] 2> 556064 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
> [junit4] 2> 556075 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.s.b.BlockDirectory Block cache on write is disabled
> [junit4] 2> 556083 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49971/solr_hdfs_home/collection1/core_node1/data
> [junit4] 2> 556115 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49971/solr_hdfs_home/collection1/core_node1/data/index
> [junit4] 2> 556138 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
> [junit4] 2> 556138 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
> [junit4] 2> 556171 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.s.b.BlockDirectory Block cache on write is disabled
> [junit4] 2> 556172 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=7, maxMergeAtOnceExplicit=2, maxMergedSegmentMB=54.2822265625, floorSegmentMB=1.00390625, forceMergeDeletesPctAllowed=4.833164813857602, segmentsPerTier=10.0, maxCFSSegmentSizeMB=0.6904296875, noCFSRatio=0.0
> [junit4] 2> 556225 INFO (IPC Server handler 2 on 49971) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:60224 <http://127.0.0.1:60224/> is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-ada032ef-beb3-4321-b044-13278d645633:NORMAL:127.0.0.1 <http://127.0.0.1/>:50603|RBW], ReplicaUC[[DISK]DS-4f0f9719-f907-44eb-a680-fe795758da6b:NORMAL:127.0.0.1:60224|RBW]]} size 0
> [junit4] 2> 556226 INFO (IPC Server handler 4 on 49971) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:50603 <http://127.0.0.1:50603/> is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-4f0f9719-f907-44eb-a680-fe795758da6b:NORMAL:127.0.0.1 <http://127.0.0.1/>:60224|RBW], ReplicaUC[[DISK]DS-e9ea3dbc-9113-4645-b893-130c3730ec1a:NORMAL:127.0.0.1:50603|FINALIZED]]} size 0
> [junit4] 2> 556233 WARN (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
> [junit4] 2> 556294 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
> [junit4] 2> 556295 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
> [junit4] 2> 556295 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
> [junit4] 2> 556307 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.u.CommitTracker Hard AutoCommit: disabled
> [junit4] 2> 556307 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.u.CommitTracker Soft AutoCommit: disabled
> [junit4] 2> 556310 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=13, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=10.5625, floorSegmentMB=0.54296875, forceMergeDeletesPctAllowed=17.113764658263925, segmentsPerTier=33.0, maxCFSSegmentSizeMB=2.0732421875, noCFSRatio=1.0
> [junit4] 2> 556316 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.s.SolrIndexSearcher Opening [Searcher@595bc441[collection1_shard2_replica_n0] main]
> [junit4] 2> 556318 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
> [junit4] 2> 556318 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
> [junit4] 2> 556319 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
> [junit4] 2> 556322 INFO (searcherExecutor-772-thread-1-processing-n:127.0.0.1:32870 <http://127.0.0.1:32870/>_ov%2Fcn x:collection1_shard2_replica_n0 s:shard2 c:collection1) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.SolrCore [collection1_shard2_replica_n0] Registered new searcher Searcher@595bc441[collection1_shard2_replica_n0] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
> [junit4] 2> 556322 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1572977170587844608
> [junit4] 2> 556330 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
> [junit4] 2> 556330 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
> [junit4] 2> 556330 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:32870/ov/cn/collection1_shard2_replica_n0/ <http://127.0.0.1:32870/ov/cn/collection1_shard2_replica_n0/>
> [junit4] 2> 556330 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
> [junit4] 2> 556330 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.SyncStrategy http://127.0.0.1:32870/ov/cn/collection1_shard2_replica_n0/ <http://127.0.0.1:32870/ov/cn/collection1_shard2_replica_n0/> has no replicas
> [junit4] 2> 556330 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
> [junit4] 2> 556340 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:32870/ov/cn/collection1_shard2_replica_n0/ <http://127.0.0.1:32870/ov/cn/collection1_shard2_replica_n0/> shard2
> [junit4] 2> 556492 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.ZkController I am the leader, no recovery necessary
> [junit4] 2> 556494 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n0&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1601
> [junit4] 2> 556498 INFO (qtp379993423-4422) [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:32870_ov%252Fcn&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2} status=0 QTime=1611
> [junit4] 2> 556650 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/shard-2-001 of type NRT
> [junit4] 2> 556650 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
> [junit4] 2> 556652 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@29097158{/ov/cn,null,AVAILABLE}
> [junit4] 2> 556652 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@d40fb8b{HTTP/1.1,[http/1.1]}{127.0.0.1:46799 <http://127.0.0.1:46799/>}
> [junit4] 2> 556653 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.e.j.s.Server Started @559546ms
> [junit4] 2> 556653 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, hostContext=/ov/cn, hostPort=46799, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/shard-2-001/cores}
> [junit4] 2> 556653 ERROR (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
> [junit4] 2> 556654 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
> [junit4] 2> 556654 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
> [junit4] 2> 556654 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
> [junit4] 2> 556654 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-07-15T08:38:48.215Z
> [junit4] 2> 556658 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
> [junit4] 2> 556658 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/shard-2-001/solr.xml
> [junit4] 2> 556664 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de, but no JMX reporters were configured - adding default JMX reporter.
> [junit4] 2> 556668 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52694/solr <http://127.0.0.1:52694/solr>
> [junit4] 2> 556685 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:46799_ov%2Fcn ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
> [junit4] 2> 556687 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:46799_ov%2Fcn ] o.a.s.c.Overseer Overseer (id=null) closing
> [junit4] 2> 556690 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:46799_ov%2Fcn ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0. <http://127.0.0.0/>1:46799_ov%2Fcn
> [junit4] 2> 556691 INFO (zkCallback-344-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
> [junit4] 2> 556692 INFO (zkCallback-349-thread-1-processing-n:127.0.0.1:32870_ov%2Fcn) [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
> [junit4] 2> 556692 INFO (zkCallback-337-thread-2-processing-n:127.0.0.1:41805_ov%2Fcn) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
> [junit4] 2> 556708 INFO (zkCallback-355-thread-1-processing-n:127.0.0.1:46799_ov%2Fcn) [n:127.0.0.1:46799_ov%2Fcn ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
> [junit4] 2> 556788 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:46799_ov%2Fcn ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
> [junit4] 2> 556794 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:46799_ov%2Fcn ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
> [junit4] 2> 556794 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:46799_ov%2Fcn ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
> [junit4] 2> 556796 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:46799_ov%2Fcn ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/shard-2-001/cores
> [junit4] 2> 556830 INFO (qtp556104417-4455) [n:127.0.0.1:46799_ov%2Fcn ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:46799_ov%252Fcn&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
> [junit4] 2> 556833 INFO (OverseerThreadFactory-758-thread-4-processing-n:127.0.0.1:41805_ov%2Fcn) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:46799_ov%2Fcn for creating new replica
> [junit4]
> --
> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> LinkedIn: http://linkedin.com/in/davidwsmiley <http://linkedin.com/in/davidwsmiley> | Book: http://www.solrenterprisesearchserver.com <http://www.solrenterprisesearchserver.com/>
Re: [JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 10 - Still unstable
Posted by David Smiley <da...@gmail.com>.
Alan, I suspect this is related the changes you did?
On Sat, Jul 15, 2017 at 6:07 AM Apache Jenkins Server <
jenkins@builds.apache.org> wrote:
> Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/10/
>
> 3 tests failed.
> FAILED: org.apache.lucene.spatial.DistanceStrategyTest.testRecipScore
> {strategy=serialized}
>
> Error Message:
>
>
> Stack Trace:
> java.lang.NullPointerException
> at
> __randomizedtesting.SeedInfo.seed([B1FC2FB143140AF1:29D25F48109F000A]:0)
> at
> org.apache.lucene.spatial.serialized.SerializedDVStrategy$ShapeDocValueSource$1.advanceExact(SerializedDVStrategy.java:186)
> at
> org.apache.lucene.spatial.util.DistanceToShapeValueSource$1.advanceExact(DistanceToShapeValueSource.java:73)
> at
> org.apache.lucene.search.DoubleValues$1.advanceExact(DoubleValues.java:53)
> at
> org.apache.lucene.spatial.util.ReciprocalDoubleValuesSource$1.advanceExact(ReciprocalDoubleValuesSource.java:57)
> at
> org.apache.lucene.spatial.StrategyTestCase.checkValueSource(StrategyTestCase.java:219)
> at
> org.apache.lucene.spatial.DistanceStrategyTest.testRecipScore(DistanceStrategyTest.java:104)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
> at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at java.lang.Thread.run(Thread.java:748)
>
>
> FAILED: org.apache.solr.update.AutoCommitTest.testMaxDocs
>
> Error Message:
> Exception during query
>
> Stack Trace:
> java.lang.RuntimeException: Exception during query
> at
> __randomizedtesting.SeedInfo.seed([793723D4943B9E3D:C0B6F50BB8D19AB7]:0)
> at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:878)
> at
> org.apache.solr.update.AutoCommitTest.testMaxDocs(AutoCommitTest.java:225)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
> at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.RuntimeException: REQUEST FAILED:
> xpath=//result[@numFound=1]
> xml response was: <?xml version="1.0" encoding="UTF-8"?>
> <response>
> <lst name="responseHeader"><int name="status">0</int><int
> name="QTime">0</int></lst><result name="response" numFound="0"
> start="0"></result>
> </response>
>
> request was:q=id:14&qt=&start=0&rows=20&version=2.2
> at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:871)
> ... 40 more
>
>
> FAILED: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test
>
> Error Message:
> Timeout waiting for all live and active
>
> Stack Trace:
> java.lang.AssertionError: Timeout waiting for all live and active
> at
> __randomizedtesting.SeedInfo.seed([793723D4943B9E3D:F1631C0E3AC7F3C5]:0)
> at org.junit.Assert.fail(Assert.java:93)
> at org.junit.Assert.assertTrue(Assert.java:43)
> at
> org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:246)
> at
> org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:136)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
> at
> org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
> at
> org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
> at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at java.lang.Thread.run(Thread.java:748)
>
>
>
>
> Build Log:
> [...truncated 9261 lines...]
> [junit4] Suite: org.apache.lucene.spatial.DistanceStrategyTest
> [junit4] 2> NOTE: download the large Jenkins line-docs file by
> running 'ant get-jenkins-line-docs' in the lucene directory.
> [junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=DistanceStrategyTest -Dtests.method=testRecipScore
> -Dtests.seed=B1FC2FB143140AF1 -Dtests.multiplier=2 -Dtests.nightly=true
> -Dtests.slow=true
> -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt
> -Dtests.locale=ar-AE -Dtests.timezone=Pacific/Chatham -Dtests.asserts=true
> -Dtests.file.encoding=US-ASCII
> [junit4] ERROR 0.03s J0 | DistanceStrategyTest.testRecipScore
> {strategy=serialized} <<<
> [junit4] > Throwable #1: java.lang.NullPointerException
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([B1FC2FB143140AF1:29D25F48109F000A]:0)
> [junit4] > at
> org.apache.lucene.spatial.serialized.SerializedDVStrategy$ShapeDocValueSource$1.advanceExact(SerializedDVStrategy.java:186)
> [junit4] > at
> org.apache.lucene.spatial.util.DistanceToShapeValueSource$1.advanceExact(DistanceToShapeValueSource.java:73)
> [junit4] > at
> org.apache.lucene.search.DoubleValues$1.advanceExact(DoubleValues.java:53)
> [junit4] > at
> org.apache.lucene.spatial.util.ReciprocalDoubleValuesSource$1.advanceExact(ReciprocalDoubleValuesSource.java:57)
> [junit4] > at
> org.apache.lucene.spatial.StrategyTestCase.checkValueSource(StrategyTestCase.java:219)
> [junit4] > at
> org.apache.lucene.spatial.DistanceStrategyTest.testRecipScore(DistanceStrategyTest.java:104)
> [junit4] > at java.lang.Thread.run(Thread.java:748)
> [junit4] 2> NOTE: test params are: codec=Asserting(Lucene70):
> {recursive_packedquad=PostingsFormat(name=MockRandom),
> recursive_quad=PostingsFormat(name=MockRandom),
> bbox__xdl=BlockTreeOrds(blocksize=128),
> termquery_geohash=PostingsFormat(name=MockRandom),
> id=BlockTreeOrds(blocksize=128)},
> docValues:{bbox__maxX=DocValuesFormat(name=Direct),
> bbox__maxY=DocValuesFormat(name=Lucene70),
> bbox__minX=DocValuesFormat(name=Asserting),
> bbox__minY=DocValuesFormat(name=Memory),
> pointvector__y=DocValuesFormat(name=Lucene70),
> pointvector__x=DocValuesFormat(name=Direct),
> serialized=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=1913,
> maxMBSortInHeap=5.545002168373545, sim=RandomSimilarity(queryNorm=true):
> {}, locale=ar-AE, timezone=Pacific/Chatham
> [junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation
> 1.8.0_131 (64-bit)/cpus=4,threads=1,free=291830768,total=324534272
> [junit4] 2> NOTE: All tests run in this JVM: [SpatialPrefixTreeTest,
> NumberRangeFacetsTest, TestTermQueryPrefixGridStrategy,
> TestRecursivePrefixTreeStrategy, Geo3dShapeSphereModelRectRelationTest,
> DistanceStrategyTest]
> [junit4] Completed [12/23 (1!)] on J0 in 0.69s, 12 tests, 1 error <<<
> FAILURES!
>
> [...truncated 2297 lines...]
> [junit4] Suite: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest
> [junit4] 2> Creating dataDir:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/init-core-data-001
> [junit4] 2> 548632 WARN
> (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [
> ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=38 numCloses=38
> [junit4] 2> 548632 INFO
> (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [
> ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true)
> w/NUMERIC_DOCVALUES_SYSPROP=true
> [junit4] 2> 548634 INFO
> (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [
> ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via:
> @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
> [junit4] 2> 548634 INFO
> (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [
> ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property:
> /ov/cn
> [junit4] 1> Formatting using clusterid: testClusterID
> [junit4] 2> 548849 WARN
> (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [
> ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried
> hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
> [junit4] 2> 548855 WARN
> (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [
> ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
> [junit4] 2> 548857 INFO
> (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [
> ] o.m.log jetty-6.1.26
> [junit4] 2> 548870 INFO
> (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [
> ] o.m.log Extract
> jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs
> to ./temp/Jetty_localhost_56321_hdfs____.206o6d/webapp
> [junit4] 2> 549299 INFO
> (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [
> ] o.m.log Started
> HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:56321
> [junit4] 2> 549422 WARN
> (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [
> ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
> [junit4] 2> 549423 INFO
> (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [
> ] o.m.log jetty-6.1.26
> [junit4] 2> 549436 INFO
> (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [
> ] o.m.log Extract
> jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode
> to ./temp/Jetty_localhost_59355_datanode____.vqlzu2/webapp
> [junit4] 2> 549878 INFO
> (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [
> ] o.m.log Started
> HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:59355
> [junit4] 2> 549915 WARN
> (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [
> ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
> [junit4] 2> 549916 INFO
> (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [
> ] o.m.log jetty-6.1.26
> [junit4] 2> 549934 INFO
> (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [
> ] o.m.log Extract
> jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode
> to ./temp/Jetty_localhost_54759_datanode____jrk8hv/webapp
> [junit4] 2> 550164 INFO (IPC Server handler 4 on 49971) [ ]
> BlockStateChange BLOCK* processReport: from storage
> DS-e9ea3dbc-9113-4645-b893-130c3730ec1a node DatanodeRegistration(
> 127.0.0.1:50603, datanodeUuid=527ac2ae-b401-4d84-86e3-6f935dee3843,
> infoPort=54653, infoSecurePort=0, ipcPort=36188,
> storageInfo=lv=-56;cid=testClusterID;nsid=451575161;c=0), blocks: 0,
> hasStaleStorage: true, processing time: 0 msecs
> [junit4] 2> 550164 INFO (IPC Server handler 4 on 49971) [ ]
> BlockStateChange BLOCK* processReport: from storage
> DS-ada032ef-beb3-4321-b044-13278d645633 node DatanodeRegistration(
> 127.0.0.1:50603, datanodeUuid=527ac2ae-b401-4d84-86e3-6f935dee3843,
> infoPort=54653, infoSecurePort=0, ipcPort=36188,
> storageInfo=lv=-56;cid=testClusterID;nsid=451575161;c=0), blocks: 0,
> hasStaleStorage: false, processing time: 0 msecs
> [junit4] 2> 550420 INFO
> (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [
> ] o.m.log Started
> HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:54759
> [junit4] 2> 550918 INFO (IPC Server handler 9 on 49971) [ ]
> BlockStateChange BLOCK* processReport: from storage
> DS-7b441025-265d-40cb-ac06-5e2d91c89e3b node DatanodeRegistration(
> 127.0.0.1:60224, datanodeUuid=48e239a8-7fb1-4b7a-bad7-3965b9821989,
> infoPort=45761, infoSecurePort=0, ipcPort=36307,
> storageInfo=lv=-56;cid=testClusterID;nsid=451575161;c=0), blocks: 0,
> hasStaleStorage: true, processing time: 0 msecs
> [junit4] 2> 550918 INFO (IPC Server handler 9 on 49971) [ ]
> BlockStateChange BLOCK* processReport: from storage
> DS-4f0f9719-f907-44eb-a680-fe795758da6b node DatanodeRegistration(
> 127.0.0.1:60224, datanodeUuid=48e239a8-7fb1-4b7a-bad7-3965b9821989,
> infoPort=45761, infoSecurePort=0, ipcPort=36307,
> storageInfo=lv=-56;cid=testClusterID;nsid=451575161;c=0), blocks: 0,
> hasStaleStorage: false, processing time: 0 msecs
> [junit4] 2> 551053 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
> [junit4] 2> 551060 INFO (Thread-2097) [ ] o.a.s.c.ZkTestServer
> client port:0.0.0.0/0.0.0.0:0
> [junit4] 2> 551060 INFO (Thread-2097) [ ] o.a.s.c.ZkTestServer
> Starting server
> [junit4] 2> 551075 ERROR (Thread-2097) [ ] o.a.z.s.ZooKeeperServer
> ZKShutdownHandler is not registered, so ZooKeeper server won't take any
> action on ERROR or SHUTDOWN server state changes
> [junit4] 2> 551160 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.c.ZkTestServer start zk server on port:52694
> [junit4] 2> 551179 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.c.AbstractZkTestCase put
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml
> to /configs/conf1/solrconfig.xml
> [junit4] 2> 551181 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.c.AbstractZkTestCase put
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml
> to /configs/conf1/schema.xml
> [junit4] 2> 551183 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.c.AbstractZkTestCase put
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
> to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
> [junit4] 2> 551185 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.c.AbstractZkTestCase put
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
> to /configs/conf1/stopwords.txt
> [junit4] 2> 551187 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.c.AbstractZkTestCase put
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt
> to /configs/conf1/protwords.txt
> [junit4] 2> 551189 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.c.AbstractZkTestCase put
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml
> to /configs/conf1/currency.xml
> [junit4] 2> 551191 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.c.AbstractZkTestCase put
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
> to /configs/conf1/enumsConfig.xml
> [junit4] 2> 551192 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.c.AbstractZkTestCase put
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
> to /configs/conf1/open-exchange-rates.json
> [junit4] 2> 551194 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.c.AbstractZkTestCase put
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
> to /configs/conf1/mapping-ISOLatin1Accent.txt
> [junit4] 2> 551196 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.c.AbstractZkTestCase put
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
> to /configs/conf1/old_synonyms.txt
> [junit4] 2> 551198 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.c.AbstractZkTestCase put
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
> to /configs/conf1/synonyms.txt
> [junit4] 2> 551201 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless
> explicitly asked otherwise
> [junit4] 2> 551336 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.e.j.s.Server jetty-9.3.14.v20161028
> [junit4] 2> 551338 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@16cae510
> {/ov/cn,null,AVAILABLE}
> [junit4] 2> 551338 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.e.j.s.AbstractConnector Started ServerConnector@597fe4f4
> {HTTP/1.1,[http/1.1]}{127.0.0.1:41805}
> [junit4] 2> 551338 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.e.j.s.Server Started @554231ms
> [junit4] 2> 551338 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/ov/cn,
> hostPort=41805,
> coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/control-001/cores}
> [junit4] 2> 551338 ERROR
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may
> be missing or incomplete.
> [junit4] 2> 551340 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr?
> version 7.1.0
> [junit4] 2> 551340 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port
> null
> [junit4] 2> 551340 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default
> config dir: null
> [junit4] 2> 551340 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time:
> 2017-07-15T08:38:42.901Z
> [junit4] 2> 551346 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in
> ZooKeeper)
> [junit4] 2> 551346 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.c.SolrXmlConfig Loading container configuration from
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/control-001/solr.xml
> [junit4] 2> 551352 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.c.SolrXmlConfig MBean server found:
> com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de, but no JMX reporters
> were configured - adding default JMX reporter.
> [junit4] 2> 551355 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52694/solr
> [junit4] 2> 551468 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D])
> [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.Overseer Overseer (id=null)
> closing
> [junit4] 2> 551469 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D])
> [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.OverseerElectionContext I am
> going to be the leader 127.0.0.1:41805_ov%2Fcn
> [junit4] 2> 551470 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D])
> [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.Overseer Overseer
> (id=98311072817348612-127.0.0.1:41805_ov%2Fcn-n_0000000000) starting
> [junit4] 2> 551480 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D])
> [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.ZkController Register node as
> live in ZooKeeper:/live_nodes/127.0.0.1:41805_ov%2Fcn
> [junit4] 2> 551482 INFO
> (zkCallback-337-thread-1-processing-n:127.0.0.1:41805_ov%2Fcn)
> [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.c.ZkStateReader Updated live
> nodes from ZooKeeper... (0) -> (1)
> [junit4] 2> 551605 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D])
> [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.m.r.SolrJmxReporter JMX monitoring
> for 'solr.node' (registry 'solr.node') enabled at server:
> com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
> [junit4] 2> 551613 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D])
> [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.m.r.SolrJmxReporter JMX monitoring
> for 'solr.jvm' (registry 'solr.jvm') enabled at server:
> com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
> [junit4] 2> 551613 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D])
> [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.m.r.SolrJmxReporter JMX monitoring
> for 'solr.jetty' (registry 'solr.jetty') enabled at server:
> com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
> [junit4] 2> 551615 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D])
> [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.CorePropertiesLocator Found 0
> core definitions underneath
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/control-001/cores
> [junit4] 2> 551639 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
> [junit4] 2> 551640 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:52694/solr
> ready
> [junit4] 2> 551642 INFO (qtp1213643546-4367) [n:127.0.0.1:41805_ov%2Fcn
> ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with
> params
> replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:41805_ov%252Fcn&wt=javabin&version=2
> and sendToOCPQueue=true
> [junit4] 2> 551645 INFO
> (OverseerThreadFactory-758-thread-1-processing-n:127.0.0.1:41805_ov%2Fcn)
> [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.CreateCollectionCmd Create
> collection control_collection
> [junit4] 2> 551750 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> ] o.a.s.h.a.CoreAdminOperation core create command
> qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
> [junit4] 2> 551751 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4
> transient cores
> [junit4] 2> 551858 INFO
> (zkCallback-337-thread-1-processing-n:127.0.0.1:41805_ov%2Fcn)
> [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.c.ZkStateReader A cluster state
> change: [WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/collections/control_collection/state.json] for collection
> [control_collection] has occurred - updating... (live nodes size: [1])
> [junit4] 2> 552773 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
> [junit4] 2> 552787 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
> [junit4] 2> 552875 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
> [junit4] 2> 552895 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.c.CoreContainer Creating SolrCore
> 'control_collection_shard1_replica_n1' using configuration from collection
> control_collection, trusted=true
> [junit4] 2> 552895 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.m.r.SolrJmxReporter JMX monitoring for
> 'solr.core.control_collection.shard1.replica_n1' (registry
> 'solr.core.control_collection.shard1.replica_n1') enabled at server:
> com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
> [junit4] 2> 552895 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.c.HdfsDirectoryFactory
> solr.hdfs.home=hdfs://localhost:49971/solr_hdfs_home
> [junit4] 2> 552895 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
> [junit4] 2> 552895 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
> [junit4] 2> 552896 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new
> SolrCore at
> [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/control-001/cores/control_collection_shard1_replica_n1],
> dataDir=[hdfs://localhost:49971/solr_hdfs_home/control_collection/core_node1/data/]
> [junit4] 2> 552907 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.c.HdfsDirectoryFactory creating directory factory for path
> hdfs://localhost:49971/solr_hdfs_home/control_collection/core_node1/data/snapshot_metadata
> [junit4] 2> 552916 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct
> memory allocation set to [true]
> [junit4] 2> 552916 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of
> [4194304] will allocate [1] slabs and use ~[4194304] bytes
> [junit4] 2> 552917 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.c.HdfsDirectoryFactory Creating new global HDFS BlockCache
> [junit4] 2> 552925 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.s.b.BlockDirectory Block cache on write is disabled
> [junit4] 2> 552926 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.c.HdfsDirectoryFactory creating directory factory for path
> hdfs://localhost:49971/solr_hdfs_home/control_collection/core_node1/data
> [junit4] 2> 552951 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.c.HdfsDirectoryFactory creating directory factory for path
> hdfs://localhost:49971/solr_hdfs_home/control_collection/core_node1/data/index
> [junit4] 2> 552958 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct
> memory allocation set to [true]
> [junit4] 2> 552958 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of
> [4194304] will allocate [1] slabs and use ~[4194304] bytes
> [junit4] 2> 552963 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.s.b.BlockDirectory Block cache on write is disabled
> [junit4] 2> 552964 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class
> org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy:
> maxMergeAtOnce=7, maxMergeAtOnceExplicit=2,
> maxMergedSegmentMB=54.2822265625, floorSegmentMB=1.00390625,
> forceMergeDeletesPctAllowed=4.833164813857602, segmentsPerTier=10.0,
> maxCFSSegmentSizeMB=0.6904296875, noCFSRatio=0.0
> [junit4] 2> 553000 INFO (IPC Server handler 9 on 49971) [ ]
> BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:60224
> is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION,
> truncateBlock=null, primaryNodeIndex=-1,
> replicas=[ReplicaUC[[DISK]DS-e9ea3dbc-9113-4645-b893-130c3730ec1a:NORMAL:
> 127.0.0.1:50603|RBW],
> ReplicaUC[[DISK]DS-7b441025-265d-40cb-ac06-5e2d91c89e3b:NORMAL:127.0.0.1:60224|FINALIZED]]}
> size 0
> [junit4] 2> 553007 INFO (IPC Server handler 3 on 49971) [ ]
> BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:50603
> is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION,
> truncateBlock=null, primaryNodeIndex=-1,
> replicas=[ReplicaUC[[DISK]DS-7b441025-265d-40cb-ac06-5e2d91c89e3b:NORMAL:
> 127.0.0.1:60224|FINALIZED],
> ReplicaUC[[DISK]DS-ada032ef-beb3-4321-b044-13278d645633:NORMAL:127.0.0.1:50603|FINALIZED]]}
> size 0
> [junit4] 2> 553015 WARN (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type =
> requestHandler,name = /dump,class = DumpRequestHandler,attributes =
> {initParams=a, name=/dump, class=DumpRequestHandler},args =
> {defaults={a=A,b=B}}}
> [junit4] 2> 553071 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.u.UpdateHandler Using UpdateLog implementation:
> org.apache.solr.update.HdfsUpdateLog
> [junit4] 2> 553071 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH
> numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
> [junit4] 2> 553071 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
> [junit4] 2> 553096 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.u.CommitTracker Hard AutoCommit: disabled
> [junit4] 2> 553096 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.u.CommitTracker Soft AutoCommit: disabled
> [junit4] 2> 553103 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class
> org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy:
> maxMergeAtOnce=13, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=10.5625,
> floorSegmentMB=0.54296875, forceMergeDeletesPctAllowed=17.113764658263925,
> segmentsPerTier=33.0, maxCFSSegmentSizeMB=2.0732421875, noCFSRatio=1.0
> [junit4] 2> 553112 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.s.SolrIndexSearcher Opening [Searcher@144f5f33[control_collection_shard1_replica_n1]
> main]
> [junit4] 2> 553123 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with
> znodeBase: /configs/conf1
> [junit4] 2> 553124 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using
> ZooKeeperStorageIO:path=/configs/conf1
> [junit4] 2> 553125 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.h.ReplicationHandler Commits will be reserved for 10000
> [junit4] 2> 553127 INFO (searcherExecutor-761-thread-1-processing-n:
> 127.0.0.1:41805_ov%2Fcn x:control_collection_shard1_replica_n1 s:shard1
> c:control_collection) [n:127.0.0.1:41805_ov%2Fcn c:control_collection
> s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore
> [control_collection_shard1_replica_n1] Registered new searcher
> Searcher@144f5f33[control_collection_shard1_replica_n1]
> main{ExitableDirectoryReader(UninvertingDirectoryReader())}
> [junit4] 2> 553136 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.u.UpdateLog Could not find max version in index or recent updates,
> using new clock 1572977167247081472
> [junit4] 2> 553144 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
> [junit4] 2> 553144 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
> [junit4] 2> 553144 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.c.SyncStrategy Sync replicas to
> http://127.0.0.1:41805/ov/cn/control_collection_shard1_replica_n1/
> [junit4] 2> 553144 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
> [junit4] 2> 553144 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.c.SyncStrategy
> http://127.0.0.1:41805/ov/cn/control_collection_shard1_replica_n1/ has no
> replicas
> [junit4] 2> 553144 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.c.ShardLeaderElectionContext Found all replicas participating in
> election, clear LIR
> [junit4] 2> 553172 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.c.ShardLeaderElectionContext I am the new leader:
> http://127.0.0.1:41805/ov/cn/control_collection_shard1_replica_n1/ shard1
> [junit4] 2> 553274 INFO
> (zkCallback-337-thread-1-processing-n:127.0.0.1:41805_ov%2Fcn)
> [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.c.ZkStateReader A cluster state
> change: [WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/collections/control_collection/state.json] for collection
> [control_collection] has occurred - updating... (live nodes size: [1])
> [junit4] 2> 553324 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.c.ZkController I am the leader, no recovery necessary
> [junit4] 2> 553326 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn
> c:control_collection s:shard1 x:control_collection_shard1_replica_n1]
> o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores
> params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT}
> status=0 QTime=1575
> [junit4] 2> 553333 INFO (qtp1213643546-4367) [n:127.0.0.1:41805_ov%2Fcn
> ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for
> at most 30 seconds. Check all shard replicas
> [junit4] 2> 553435 INFO
> (zkCallback-337-thread-1-processing-n:127.0.0.1:41805_ov%2Fcn)
> [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.c.ZkStateReader A cluster state
> change: [WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/collections/control_collection/state.json] for collection
> [control_collection] has occurred - updating... (live nodes size: [1])
> [junit4] 2> 553649 INFO
> (OverseerCollectionConfigSetProcessor-98311072817348612-127.0.0.1:41805_ov%2Fcn-n_0000000000)
> [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.OverseerTaskQueue Response ZK
> path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.
> Requestor may have disconnected from ZooKeeper
> [junit4] 2> 554333 INFO (qtp1213643546-4367) [n:127.0.0.1:41805_ov%2Fcn
> ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections
> params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:41805_ov%252Fcn&wt=javabin&version=2}
> status=0 QTime=2691
> [junit4] 2> 554341 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
> [junit4] 2> 554342 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:52694/solr
> ready
> [junit4] 2> 554342 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection
> loss:false
> [junit4] 2> 554343 INFO (qtp1213643546-4368) [n:127.0.0.1:41805_ov%2Fcn
> ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with
> params
> replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2
> and sendToOCPQueue=true
> [junit4] 2> 554345 INFO
> (OverseerThreadFactory-758-thread-2-processing-n:127.0.0.1:41805_ov%2Fcn)
> [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.CreateCollectionCmd Create
> collection collection1
> [junit4] 2> 554347 WARN
> (OverseerThreadFactory-758-thread-2-processing-n:127.0.0.1:41805_ov%2Fcn)
> [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.CreateCollectionCmd It is unusual
> to create a collection (collection1) without cores.
> [junit4] 2> 554558 INFO (qtp1213643546-4368) [n:127.0.0.1:41805_ov%2Fcn
> ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for
> at most 30 seconds. Check all shard replicas
> [junit4] 2> 554558 INFO (qtp1213643546-4368) [n:127.0.0.1:41805_ov%2Fcn
> ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections
> params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2}
> status=0 QTime=214
> [junit4] 2> 554704 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/shard-1-001
> of type NRT
> [junit4] 2> 554705 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.e.j.s.Server jetty-9.3.14.v20161028
> [junit4] 2> 554707 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1ee5d6f8
> {/ov/cn,null,AVAILABLE}
> [junit4] 2> 554708 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.e.j.s.AbstractConnector Started ServerConnector@2bdfb805
> {HTTP/1.1,[http/1.1]}{127.0.0.1:32870}
> [junit4] 2> 554708 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.e.j.s.Server Started @557601ms
> [junit4] 2> 554708 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml,
> hostContext=/ov/cn, hostPort=32870,
> coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/shard-1-001/cores}
> [junit4] 2> 554708 ERROR
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may
> be missing or incomplete.
> [junit4] 2> 554708 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr?
> version 7.1.0
> [junit4] 2> 554708 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port
> null
> [junit4] 2> 554708 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default
> config dir: null
> [junit4] 2> 554708 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time:
> 2017-07-15T08:38:46.269Z
> [junit4] 2> 554722 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in
> ZooKeeper)
> [junit4] 2> 554722 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.c.SolrXmlConfig Loading container configuration from
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/shard-1-001/solr.xml
> [junit4] 2> 554729 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.c.SolrXmlConfig MBean server found:
> com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de, but no JMX reporters
> were configured - adding default JMX reporter.
> [junit4] 2> 554732 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52694/solr
> [junit4] 2> 554746 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D])
> [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.c.c.ZkStateReader Updated live
> nodes from ZooKeeper... (0) -> (1)
> [junit4] 2> 554749 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D])
> [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.c.Overseer Overseer (id=null)
> closing
> [junit4] 2> 554752 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D])
> [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.c.ZkController Register node as
> live in ZooKeeper:/live_nodes/127.0.0.1:32870_ov%2Fcn
> [junit4] 2> 554753 INFO (zkCallback-344-thread-1) [ ]
> o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
> [junit4] 2> 554755 INFO
> (zkCallback-337-thread-1-processing-n:127.0.0.1:41805_ov%2Fcn)
> [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.c.ZkStateReader Updated live
> nodes from ZooKeeper... (1) -> (2)
> [junit4] 2> 554760 INFO
> (zkCallback-349-thread-1-processing-n:127.0.0.1:32870_ov%2Fcn)
> [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.c.c.ZkStateReader Updated live
> nodes from ZooKeeper... (1) -> (2)
> [junit4] 2> 554850 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D])
> [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.m.r.SolrJmxReporter JMX monitoring
> for 'solr.node' (registry 'solr.node') enabled at server:
> com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
> [junit4] 2> 554856 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D])
> [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.m.r.SolrJmxReporter JMX monitoring
> for 'solr.jvm' (registry 'solr.jvm') enabled at server:
> com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
> [junit4] 2> 554856 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D])
> [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.m.r.SolrJmxReporter JMX monitoring
> for 'solr.jetty' (registry 'solr.jetty') enabled at server:
> com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
> [junit4] 2> 554857 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D])
> [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.c.CorePropertiesLocator Found 0
> core definitions underneath
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/shard-1-001/cores
> [junit4] 2> 554887 INFO (qtp379993423-4422) [n:127.0.0.1:32870_ov%2Fcn
> ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with
> params node=127.0.0.1:32870_ov%252Fcn&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2
> and sendToOCPQueue=true
> [junit4] 2> 554890 INFO
> (OverseerThreadFactory-758-thread-3-processing-n:127.0.0.1:41805_ov%2Fcn)
> [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.AddReplicaCmd Node Identified
> 127.0.0.1:32870_ov%2Fcn for creating new replica
> [junit4] 2> 554890 INFO
> (OverseerCollectionConfigSetProcessor-98311072817348612-127.0.0.1:41805_ov%2Fcn-n_0000000000)
> [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.OverseerTaskQueue Response ZK
> path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.
> Requestor may have disconnected from ZooKeeper
> [junit4] 2> 554893 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> ] o.a.s.h.a.CoreAdminOperation core create command
> qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n0&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
> [junit4] 2> 554893 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4
> transient cores
> [junit4] 2> 555915 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.SolrConfig
> Using Lucene MatchVersion: 7.1.0
> [junit4] 2> 555932 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.s.IndexSchema [collection1_shard2_replica_n0] Schema name=test
> [junit4] 2> 556029 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
> [junit4] 2> 556051 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n0'
> using configuration from collection collection1, trusted=true
> [junit4] 2> 556052 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.m.r.SolrJmxReporter JMX monitoring for
> 'solr.core.collection1.shard2.replica_n0' (registry
> 'solr.core.collection1.shard2.replica_n0') enabled at server:
> com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
> [junit4] 2> 556052 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.c.HdfsDirectoryFactory
> solr.hdfs.home=hdfs://localhost:49971/solr_hdfs_home
> [junit4] 2> 556052 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
> [junit4] 2> 556052 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.SolrCore
> solr.RecoveryStrategy.Builder
> [junit4] 2> 556052 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.SolrCore
> [[collection1_shard2_replica_n0] ] Opening new SolrCore at
> [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/shard-1-001/cores/collection1_shard2_replica_n0],
> dataDir=[hdfs://localhost:49971/solr_hdfs_home/collection1/core_node1/data/]
> [junit4] 2> 556054 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.c.HdfsDirectoryFactory creating directory factory for path
> hdfs://localhost:49971/solr_hdfs_home/collection1/core_node1/data/snapshot_metadata
> [junit4] 2> 556064 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct
> memory allocation set to [true]
> [junit4] 2> 556064 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of
> [4194304] will allocate [1] slabs and use ~[4194304] bytes
> [junit4] 2> 556075 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.s.b.BlockDirectory Block cache on write is disabled
> [junit4] 2> 556083 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.c.HdfsDirectoryFactory creating directory factory for path
> hdfs://localhost:49971/solr_hdfs_home/collection1/core_node1/data
> [junit4] 2> 556115 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.c.HdfsDirectoryFactory creating directory factory for path
> hdfs://localhost:49971/solr_hdfs_home/collection1/core_node1/data/index
> [junit4] 2> 556138 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct
> memory allocation set to [true]
> [junit4] 2> 556138 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of
> [4194304] will allocate [1] slabs and use ~[4194304] bytes
> [junit4] 2> 556171 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.s.b.BlockDirectory Block cache on write is disabled
> [junit4] 2> 556172 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class
> org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy:
> maxMergeAtOnce=7, maxMergeAtOnceExplicit=2,
> maxMergedSegmentMB=54.2822265625, floorSegmentMB=1.00390625,
> forceMergeDeletesPctAllowed=4.833164813857602, segmentsPerTier=10.0,
> maxCFSSegmentSizeMB=0.6904296875, noCFSRatio=0.0
> [junit4] 2> 556225 INFO (IPC Server handler 2 on 49971) [ ]
> BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:60224
> is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION,
> truncateBlock=null, primaryNodeIndex=-1,
> replicas=[ReplicaUC[[DISK]DS-ada032ef-beb3-4321-b044-13278d645633:NORMAL:
> 127.0.0.1:50603|RBW],
> ReplicaUC[[DISK]DS-4f0f9719-f907-44eb-a680-fe795758da6b:NORMAL:127.0.0.1:60224|RBW]]}
> size 0
> [junit4] 2> 556226 INFO (IPC Server handler 4 on 49971) [ ]
> BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:50603
> is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION,
> truncateBlock=null, primaryNodeIndex=-1,
> replicas=[ReplicaUC[[DISK]DS-4f0f9719-f907-44eb-a680-fe795758da6b:NORMAL:
> 127.0.0.1:60224|RBW],
> ReplicaUC[[DISK]DS-e9ea3dbc-9113-4645-b893-130c3730ec1a:NORMAL:127.0.0.1:50603|FINALIZED]]}
> size 0
> [junit4] 2> 556233 WARN (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type =
> requestHandler,name = /dump,class = DumpRequestHandler,attributes =
> {initParams=a, name=/dump, class=DumpRequestHandler},args =
> {defaults={a=A,b=B}}}
> [junit4] 2> 556294 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.u.UpdateHandler Using UpdateLog implementation:
> org.apache.solr.update.HdfsUpdateLog
> [junit4] 2> 556295 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.u.UpdateLog
> Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH
> numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
> [junit4] 2> 556295 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
> [junit4] 2> 556307 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.u.CommitTracker Hard AutoCommit: disabled
> [junit4] 2> 556307 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.u.CommitTracker Soft AutoCommit: disabled
> [junit4] 2> 556310 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class
> org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy:
> maxMergeAtOnce=13, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=10.5625,
> floorSegmentMB=0.54296875, forceMergeDeletesPctAllowed=17.113764658263925,
> segmentsPerTier=33.0, maxCFSSegmentSizeMB=2.0732421875, noCFSRatio=1.0
> [junit4] 2> 556316 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.s.SolrIndexSearcher Opening [Searcher@595bc441[collection1_shard2_replica_n0]
> main]
> [junit4] 2> 556318 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with
> znodeBase: /configs/conf1
> [junit4] 2> 556318 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using
> ZooKeeperStorageIO:path=/configs/conf1
> [junit4] 2> 556319 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.h.ReplicationHandler Commits will be reserved for 10000
> [junit4] 2> 556322 INFO (searcherExecutor-772-thread-1-processing-n:
> 127.0.0.1:32870_ov%2Fcn x:collection1_shard2_replica_n0 s:shard2
> c:collection1) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2
> x:collection1_shard2_replica_n0] o.a.s.c.SolrCore
> [collection1_shard2_replica_n0] Registered new searcher Searcher@595bc441[collection1_shard2_replica_n0]
> main{ExitableDirectoryReader(UninvertingDirectoryReader())}
> [junit4] 2> 556322 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.u.UpdateLog
> Could not find max version in index or recent updates, using new clock
> 1572977170587844608
> [junit4] 2> 556330 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
> [junit4] 2> 556330 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
> [junit4] 2> 556330 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.c.SyncStrategy Sync replicas to
> http://127.0.0.1:32870/ov/cn/collection1_shard2_replica_n0/
> [junit4] 2> 556330 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
> [junit4] 2> 556330 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.c.SyncStrategy
> http://127.0.0.1:32870/ov/cn/collection1_shard2_replica_n0/ has no
> replicas
> [junit4] 2> 556330 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.c.ShardLeaderElectionContext Found all replicas participating in
> election, clear LIR
> [junit4] 2> 556340 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.c.ShardLeaderElectionContext I am the new leader:
> http://127.0.0.1:32870/ov/cn/collection1_shard2_replica_n0/ shard2
> [junit4] 2> 556492 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.c.ZkController I am the leader, no recovery necessary
> [junit4] 2> 556494 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn
> c:collection1 s:shard2 x:collection1_shard2_replica_n0]
> o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores
> params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n0&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT}
> status=0 QTime=1601
> [junit4] 2> 556498 INFO (qtp379993423-4422) [n:127.0.0.1:32870_ov%2Fcn
> ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections
> params={node=127.0.0.1:32870_ov%252Fcn&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2}
> status=0 QTime=1611
> [junit4] 2> 556650 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/shard-2-001
> of type NRT
> [junit4] 2> 556650 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.e.j.s.Server jetty-9.3.14.v20161028
> [junit4] 2> 556652 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@29097158
> {/ov/cn,null,AVAILABLE}
> [junit4] 2> 556652 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.e.j.s.AbstractConnector Started ServerConnector@d40fb8b
> {HTTP/1.1,[http/1.1]}{127.0.0.1:46799}
> [junit4] 2> 556653 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.e.j.s.Server Started @559546ms
> [junit4] 2> 556653 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml,
> hostContext=/ov/cn, hostPort=46799,
> coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/shard-2-001/cores}
> [junit4] 2> 556653 ERROR
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may
> be missing or incomplete.
> [junit4] 2> 556654 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr?
> version 7.1.0
> [junit4] 2> 556654 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port
> null
> [junit4] 2> 556654 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default
> config dir: null
> [junit4] 2> 556654 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time:
> 2017-07-15T08:38:48.215Z
> [junit4] 2> 556658 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in
> ZooKeeper)
> [junit4] 2> 556658 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.c.SolrXmlConfig Loading container configuration from
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/shard-2-001/solr.xml
> [junit4] 2> 556664 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.c.SolrXmlConfig MBean server found:
> com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de, but no JMX reporters
> were configured - adding default JMX reporter.
> [junit4] 2> 556668 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ]
> o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52694/solr
> [junit4] 2> 556685 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D])
> [n:127.0.0.1:46799_ov%2Fcn ] o.a.s.c.c.ZkStateReader Updated live
> nodes from ZooKeeper... (0) -> (2)
> [junit4] 2> 556687 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D])
> [n:127.0.0.1:46799_ov%2Fcn ] o.a.s.c.Overseer Overseer (id=null)
> closing
> [junit4] 2> 556690 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D])
> [n:127.0.0.1:46799_ov%2Fcn ] o.a.s.c.ZkController Register node as
> live in ZooKeeper:/live_nodes/127.0.0.1:46799_ov%2Fcn
> [junit4] 2> 556691 INFO (zkCallback-344-thread-1) [ ]
> o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
> [junit4] 2> 556692 INFO
> (zkCallback-349-thread-1-processing-n:127.0.0.1:32870_ov%2Fcn)
> [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.c.c.ZkStateReader Updated live
> nodes from ZooKeeper... (2) -> (3)
> [junit4] 2> 556692 INFO
> (zkCallback-337-thread-2-processing-n:127.0.0.1:41805_ov%2Fcn)
> [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.c.ZkStateReader Updated live
> nodes from ZooKeeper... (2) -> (3)
> [junit4] 2> 556708 INFO
> (zkCallback-355-thread-1-processing-n:127.0.0.1:46799_ov%2Fcn)
> [n:127.0.0.1:46799_ov%2Fcn ] o.a.s.c.c.ZkStateReader Updated live
> nodes from ZooKeeper... (2) -> (3)
> [junit4] 2> 556788 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D])
> [n:127.0.0.1:46799_ov%2Fcn ] o.a.s.m.r.SolrJmxReporter JMX monitoring
> for 'solr.node' (registry 'solr.node') enabled at server:
> com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
> [junit4] 2> 556794 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D])
> [n:127.0.0.1:46799_ov%2Fcn ] o.a.s.m.r.SolrJmxReporter JMX monitoring
> for 'solr.jvm' (registry 'solr.jvm') enabled at server:
> com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
> [junit4] 2> 556794 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D])
> [n:127.0.0.1:46799_ov%2Fcn ] o.a.s.m.r.SolrJmxReporter JMX monitoring
> for 'solr.jetty' (registry 'solr.jetty') enabled at server:
> com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
> [junit4] 2> 556796 INFO
> (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D])
> [n:127.0.0.1:46799_ov%2Fcn ] o.a.s.c.CorePropertiesLocator Found 0
> core definitions underneath
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/shard-2-001/cores
> [junit4] 2> 556830 INFO (qtp556104417-4455) [n:127.0.0.1:46799_ov%2Fcn
> ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with
> params node=127.0.0.1:46799_ov%252Fcn&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2
> and sendToOCPQueue=true
> [junit4] 2> 556833 INFO
> (OverseerThreadFactory-758-thread-4-processing-n:127.0.0.1:41805_ov%2Fcn)
> [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.AddReplicaCmd Node Identified
> 127.0.0.1:46799_ov%2Fcn for creating new replica
> [junit4]
--
Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
http://www.solrenterprisesearchserver.com
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 10 - Still
unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/10/
3 tests failed.
FAILED: org.apache.lucene.spatial.DistanceStrategyTest.testRecipScore {strategy=serialized}
Error Message:
Stack Trace:
java.lang.NullPointerException
at __randomizedtesting.SeedInfo.seed([B1FC2FB143140AF1:29D25F48109F000A]:0)
at org.apache.lucene.spatial.serialized.SerializedDVStrategy$ShapeDocValueSource$1.advanceExact(SerializedDVStrategy.java:186)
at org.apache.lucene.spatial.util.DistanceToShapeValueSource$1.advanceExact(DistanceToShapeValueSource.java:73)
at org.apache.lucene.search.DoubleValues$1.advanceExact(DoubleValues.java:53)
at org.apache.lucene.spatial.util.ReciprocalDoubleValuesSource$1.advanceExact(ReciprocalDoubleValuesSource.java:57)
at org.apache.lucene.spatial.StrategyTestCase.checkValueSource(StrategyTestCase.java:219)
at org.apache.lucene.spatial.DistanceStrategyTest.testRecipScore(DistanceStrategyTest.java:104)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.update.AutoCommitTest.testMaxDocs
Error Message:
Exception during query
Stack Trace:
java.lang.RuntimeException: Exception during query
at __randomizedtesting.SeedInfo.seed([793723D4943B9E3D:C0B6F50BB8D19AB7]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:878)
at org.apache.solr.update.AutoCommitTest.testMaxDocs(AutoCommitTest.java:225)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
</response>
request was:q=id:14&qt=&start=0&rows=20&version=2.2
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:871)
... 40 more
FAILED: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test
Error Message:
Timeout waiting for all live and active
Stack Trace:
java.lang.AssertionError: Timeout waiting for all live and active
at __randomizedtesting.SeedInfo.seed([793723D4943B9E3D:F1631C0E3AC7F3C5]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:246)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:136)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 9261 lines...]
[junit4] Suite: org.apache.lucene.spatial.DistanceStrategyTest
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=DistanceStrategyTest -Dtests.method=testRecipScore -Dtests.seed=B1FC2FB143140AF1 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=ar-AE -Dtests.timezone=Pacific/Chatham -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 0.03s J0 | DistanceStrategyTest.testRecipScore {strategy=serialized} <<<
[junit4] > Throwable #1: java.lang.NullPointerException
[junit4] > at __randomizedtesting.SeedInfo.seed([B1FC2FB143140AF1:29D25F48109F000A]:0)
[junit4] > at org.apache.lucene.spatial.serialized.SerializedDVStrategy$ShapeDocValueSource$1.advanceExact(SerializedDVStrategy.java:186)
[junit4] > at org.apache.lucene.spatial.util.DistanceToShapeValueSource$1.advanceExact(DistanceToShapeValueSource.java:73)
[junit4] > at org.apache.lucene.search.DoubleValues$1.advanceExact(DoubleValues.java:53)
[junit4] > at org.apache.lucene.spatial.util.ReciprocalDoubleValuesSource$1.advanceExact(ReciprocalDoubleValuesSource.java:57)
[junit4] > at org.apache.lucene.spatial.StrategyTestCase.checkValueSource(StrategyTestCase.java:219)
[junit4] > at org.apache.lucene.spatial.DistanceStrategyTest.testRecipScore(DistanceStrategyTest.java:104)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {recursive_packedquad=PostingsFormat(name=MockRandom), recursive_quad=PostingsFormat(name=MockRandom), bbox__xdl=BlockTreeOrds(blocksize=128), termquery_geohash=PostingsFormat(name=MockRandom), id=BlockTreeOrds(blocksize=128)}, docValues:{bbox__maxX=DocValuesFormat(name=Direct), bbox__maxY=DocValuesFormat(name=Lucene70), bbox__minX=DocValuesFormat(name=Asserting), bbox__minY=DocValuesFormat(name=Memory), pointvector__y=DocValuesFormat(name=Lucene70), pointvector__x=DocValuesFormat(name=Direct), serialized=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=1913, maxMBSortInHeap=5.545002168373545, sim=RandomSimilarity(queryNorm=true): {}, locale=ar-AE, timezone=Pacific/Chatham
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_131 (64-bit)/cpus=4,threads=1,free=291830768,total=324534272
[junit4] 2> NOTE: All tests run in this JVM: [SpatialPrefixTreeTest, NumberRangeFacetsTest, TestTermQueryPrefixGridStrategy, TestRecursivePrefixTreeStrategy, Geo3dShapeSphereModelRectRelationTest, DistanceStrategyTest]
[junit4] Completed [12/23 (1!)] on J0 in 0.69s, 12 tests, 1 error <<< FAILURES!
[...truncated 2297 lines...]
[junit4] Suite: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/init-core-data-001
[junit4] 2> 548632 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=38 numCloses=38
[junit4] 2> 548632 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 548634 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
[junit4] 2> 548634 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /ov/cn
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 548849 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 548855 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 548857 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 548870 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_56321_hdfs____.206o6d/webapp
[junit4] 2> 549299 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:56321
[junit4] 2> 549422 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 549423 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 549436 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_59355_datanode____.vqlzu2/webapp
[junit4] 2> 549878 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:59355
[junit4] 2> 549915 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 549916 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 549934 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_54759_datanode____jrk8hv/webapp
[junit4] 2> 550164 INFO (IPC Server handler 4 on 49971) [ ] BlockStateChange BLOCK* processReport: from storage DS-e9ea3dbc-9113-4645-b893-130c3730ec1a node DatanodeRegistration(127.0.0.1:50603, datanodeUuid=527ac2ae-b401-4d84-86e3-6f935dee3843, infoPort=54653, infoSecurePort=0, ipcPort=36188, storageInfo=lv=-56;cid=testClusterID;nsid=451575161;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 550164 INFO (IPC Server handler 4 on 49971) [ ] BlockStateChange BLOCK* processReport: from storage DS-ada032ef-beb3-4321-b044-13278d645633 node DatanodeRegistration(127.0.0.1:50603, datanodeUuid=527ac2ae-b401-4d84-86e3-6f935dee3843, infoPort=54653, infoSecurePort=0, ipcPort=36188, storageInfo=lv=-56;cid=testClusterID;nsid=451575161;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 550420 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[793723D4943B9E3D]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:54759
[junit4] 2> 550918 INFO (IPC Server handler 9 on 49971) [ ] BlockStateChange BLOCK* processReport: from storage DS-7b441025-265d-40cb-ac06-5e2d91c89e3b node DatanodeRegistration(127.0.0.1:60224, datanodeUuid=48e239a8-7fb1-4b7a-bad7-3965b9821989, infoPort=45761, infoSecurePort=0, ipcPort=36307, storageInfo=lv=-56;cid=testClusterID;nsid=451575161;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 550918 INFO (IPC Server handler 9 on 49971) [ ] BlockStateChange BLOCK* processReport: from storage DS-4f0f9719-f907-44eb-a680-fe795758da6b node DatanodeRegistration(127.0.0.1:60224, datanodeUuid=48e239a8-7fb1-4b7a-bad7-3965b9821989, infoPort=45761, infoSecurePort=0, ipcPort=36307, storageInfo=lv=-56;cid=testClusterID;nsid=451575161;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 551053 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 551060 INFO (Thread-2097) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 551060 INFO (Thread-2097) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 551075 ERROR (Thread-2097) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 551160 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.ZkTestServer start zk server on port:52694
[junit4] 2> 551179 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 551181 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 551183 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 551185 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 551187 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 551189 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 551191 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 551192 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 551194 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 551196 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 551198 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 551201 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 551336 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 551338 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@16cae510{/ov/cn,null,AVAILABLE}
[junit4] 2> 551338 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@597fe4f4{HTTP/1.1,[http/1.1]}{127.0.0.1:41805}
[junit4] 2> 551338 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.e.j.s.Server Started @554231ms
[junit4] 2> 551338 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/ov/cn, hostPort=41805, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/control-001/cores}
[junit4] 2> 551338 ERROR (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 551340 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 551340 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 551340 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 551340 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-07-15T08:38:42.901Z
[junit4] 2> 551346 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 551346 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/control-001/solr.xml
[junit4] 2> 551352 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 551355 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52694/solr
[junit4] 2> 551468 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 551469 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:41805_ov%2Fcn
[junit4] 2> 551470 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.Overseer Overseer (id=98311072817348612-127.0.0.1:41805_ov%2Fcn-n_0000000000) starting
[junit4] 2> 551480 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41805_ov%2Fcn
[junit4] 2> 551482 INFO (zkCallback-337-thread-1-processing-n:127.0.0.1:41805_ov%2Fcn) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 551605 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
[junit4] 2> 551613 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
[junit4] 2> 551613 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
[junit4] 2> 551615 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/control-001/cores
[junit4] 2> 551639 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 551640 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:52694/solr ready
[junit4] 2> 551642 INFO (qtp1213643546-4367) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:41805_ov%252Fcn&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 551645 INFO (OverseerThreadFactory-758-thread-1-processing-n:127.0.0.1:41805_ov%2Fcn) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 551750 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 551751 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 551858 INFO (zkCallback-337-thread-1-processing-n:127.0.0.1:41805_ov%2Fcn) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 552773 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 552787 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 552875 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 552895 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 552895 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
[junit4] 2> 552895 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:49971/solr_hdfs_home
[junit4] 2> 552895 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 552895 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 552896 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[hdfs://localhost:49971/solr_hdfs_home/control_collection/core_node1/data/]
[junit4] 2> 552907 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49971/solr_hdfs_home/control_collection/core_node1/data/snapshot_metadata
[junit4] 2> 552916 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 552916 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 552917 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Creating new global HDFS BlockCache
[junit4] 2> 552925 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 552926 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49971/solr_hdfs_home/control_collection/core_node1/data
[junit4] 2> 552951 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49971/solr_hdfs_home/control_collection/core_node1/data/index
[junit4] 2> 552958 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 552958 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 552963 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 552964 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=7, maxMergeAtOnceExplicit=2, maxMergedSegmentMB=54.2822265625, floorSegmentMB=1.00390625, forceMergeDeletesPctAllowed=4.833164813857602, segmentsPerTier=10.0, maxCFSSegmentSizeMB=0.6904296875, noCFSRatio=0.0
[junit4] 2> 553000 INFO (IPC Server handler 9 on 49971) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:60224 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-e9ea3dbc-9113-4645-b893-130c3730ec1a:NORMAL:127.0.0.1:50603|RBW], ReplicaUC[[DISK]DS-7b441025-265d-40cb-ac06-5e2d91c89e3b:NORMAL:127.0.0.1:60224|FINALIZED]]} size 0
[junit4] 2> 553007 INFO (IPC Server handler 3 on 49971) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:50603 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-7b441025-265d-40cb-ac06-5e2d91c89e3b:NORMAL:127.0.0.1:60224|FINALIZED], ReplicaUC[[DISK]DS-ada032ef-beb3-4321-b044-13278d645633:NORMAL:127.0.0.1:50603|FINALIZED]]} size 0
[junit4] 2> 553015 WARN (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 553071 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 553071 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 553071 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 553096 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 553096 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 553103 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=13, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=10.5625, floorSegmentMB=0.54296875, forceMergeDeletesPctAllowed=17.113764658263925, segmentsPerTier=33.0, maxCFSSegmentSizeMB=2.0732421875, noCFSRatio=1.0
[junit4] 2> 553112 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@144f5f33[control_collection_shard1_replica_n1] main]
[junit4] 2> 553123 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 553124 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 553125 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 553127 INFO (searcherExecutor-761-thread-1-processing-n:127.0.0.1:41805_ov%2Fcn x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@144f5f33[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 553136 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1572977167247081472
[junit4] 2> 553144 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 553144 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 553144 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:41805/ov/cn/control_collection_shard1_replica_n1/
[junit4] 2> 553144 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 553144 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:41805/ov/cn/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 553144 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 553172 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:41805/ov/cn/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 553274 INFO (zkCallback-337-thread-1-processing-n:127.0.0.1:41805_ov%2Fcn) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 553324 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 553326 INFO (qtp1213643546-4365) [n:127.0.0.1:41805_ov%2Fcn c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1575
[junit4] 2> 553333 INFO (qtp1213643546-4367) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 553435 INFO (zkCallback-337-thread-1-processing-n:127.0.0.1:41805_ov%2Fcn) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 553649 INFO (OverseerCollectionConfigSetProcessor-98311072817348612-127.0.0.1:41805_ov%2Fcn-n_0000000000) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 554333 INFO (qtp1213643546-4367) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:41805_ov%252Fcn&wt=javabin&version=2} status=0 QTime=2691
[junit4] 2> 554341 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 554342 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:52694/solr ready
[junit4] 2> 554342 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 554343 INFO (qtp1213643546-4368) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 554345 INFO (OverseerThreadFactory-758-thread-2-processing-n:127.0.0.1:41805_ov%2Fcn) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 554347 WARN (OverseerThreadFactory-758-thread-2-processing-n:127.0.0.1:41805_ov%2Fcn) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 554558 INFO (qtp1213643546-4368) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 554558 INFO (qtp1213643546-4368) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2} status=0 QTime=214
[junit4] 2> 554704 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/shard-1-001 of type NRT
[junit4] 2> 554705 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 554707 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1ee5d6f8{/ov/cn,null,AVAILABLE}
[junit4] 2> 554708 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2bdfb805{HTTP/1.1,[http/1.1]}{127.0.0.1:32870}
[junit4] 2> 554708 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.e.j.s.Server Started @557601ms
[junit4] 2> 554708 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, hostContext=/ov/cn, hostPort=32870, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/shard-1-001/cores}
[junit4] 2> 554708 ERROR (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 554708 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 554708 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 554708 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 554708 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-07-15T08:38:46.269Z
[junit4] 2> 554722 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 554722 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/shard-1-001/solr.xml
[junit4] 2> 554729 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 554732 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52694/solr
[junit4] 2> 554746 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 554749 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 554752 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32870_ov%2Fcn
[junit4] 2> 554753 INFO (zkCallback-344-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 554755 INFO (zkCallback-337-thread-1-processing-n:127.0.0.1:41805_ov%2Fcn) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 554760 INFO (zkCallback-349-thread-1-processing-n:127.0.0.1:32870_ov%2Fcn) [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 554850 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
[junit4] 2> 554856 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
[junit4] 2> 554856 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
[junit4] 2> 554857 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/shard-1-001/cores
[junit4] 2> 554887 INFO (qtp379993423-4422) [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:32870_ov%252Fcn&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 554890 INFO (OverseerThreadFactory-758-thread-3-processing-n:127.0.0.1:41805_ov%2Fcn) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:32870_ov%2Fcn for creating new replica
[junit4] 2> 554890 INFO (OverseerCollectionConfigSetProcessor-98311072817348612-127.0.0.1:41805_ov%2Fcn-n_0000000000) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 554893 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n0&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 554893 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 555915 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 555932 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.s.IndexSchema [collection1_shard2_replica_n0] Schema name=test
[junit4] 2> 556029 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 556051 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n0' using configuration from collection collection1, trusted=true
[junit4] 2> 556052 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n0' (registry 'solr.core.collection1.shard2.replica_n0') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
[junit4] 2> 556052 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:49971/solr_hdfs_home
[junit4] 2> 556052 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 556052 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 556052 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.SolrCore [[collection1_shard2_replica_n0] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/shard-1-001/cores/collection1_shard2_replica_n0], dataDir=[hdfs://localhost:49971/solr_hdfs_home/collection1/core_node1/data/]
[junit4] 2> 556054 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49971/solr_hdfs_home/collection1/core_node1/data/snapshot_metadata
[junit4] 2> 556064 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 556064 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 556075 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 556083 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49971/solr_hdfs_home/collection1/core_node1/data
[junit4] 2> 556115 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49971/solr_hdfs_home/collection1/core_node1/data/index
[junit4] 2> 556138 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 556138 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 556171 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 556172 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=7, maxMergeAtOnceExplicit=2, maxMergedSegmentMB=54.2822265625, floorSegmentMB=1.00390625, forceMergeDeletesPctAllowed=4.833164813857602, segmentsPerTier=10.0, maxCFSSegmentSizeMB=0.6904296875, noCFSRatio=0.0
[junit4] 2> 556225 INFO (IPC Server handler 2 on 49971) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:60224 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-ada032ef-beb3-4321-b044-13278d645633:NORMAL:127.0.0.1:50603|RBW], ReplicaUC[[DISK]DS-4f0f9719-f907-44eb-a680-fe795758da6b:NORMAL:127.0.0.1:60224|RBW]]} size 0
[junit4] 2> 556226 INFO (IPC Server handler 4 on 49971) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:50603 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-4f0f9719-f907-44eb-a680-fe795758da6b:NORMAL:127.0.0.1:60224|RBW], ReplicaUC[[DISK]DS-e9ea3dbc-9113-4645-b893-130c3730ec1a:NORMAL:127.0.0.1:50603|FINALIZED]]} size 0
[junit4] 2> 556233 WARN (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 556294 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 556295 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 556295 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 556307 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 556307 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 556310 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=13, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=10.5625, floorSegmentMB=0.54296875, forceMergeDeletesPctAllowed=17.113764658263925, segmentsPerTier=33.0, maxCFSSegmentSizeMB=2.0732421875, noCFSRatio=1.0
[junit4] 2> 556316 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.s.SolrIndexSearcher Opening [Searcher@595bc441[collection1_shard2_replica_n0] main]
[junit4] 2> 556318 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 556318 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 556319 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 556322 INFO (searcherExecutor-772-thread-1-processing-n:127.0.0.1:32870_ov%2Fcn x:collection1_shard2_replica_n0 s:shard2 c:collection1) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.SolrCore [collection1_shard2_replica_n0] Registered new searcher Searcher@595bc441[collection1_shard2_replica_n0] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 556322 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1572977170587844608
[junit4] 2> 556330 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 556330 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 556330 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:32870/ov/cn/collection1_shard2_replica_n0/
[junit4] 2> 556330 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 556330 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.SyncStrategy http://127.0.0.1:32870/ov/cn/collection1_shard2_replica_n0/ has no replicas
[junit4] 2> 556330 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 556340 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:32870/ov/cn/collection1_shard2_replica_n0/ shard2
[junit4] 2> 556492 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 556494 INFO (qtp379993423-4418) [n:127.0.0.1:32870_ov%2Fcn c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n0&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1601
[junit4] 2> 556498 INFO (qtp379993423-4422) [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:32870_ov%252Fcn&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2} status=0 QTime=1611
[junit4] 2> 556650 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/shard-2-001 of type NRT
[junit4] 2> 556650 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 556652 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@29097158{/ov/cn,null,AVAILABLE}
[junit4] 2> 556652 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@d40fb8b{HTTP/1.1,[http/1.1]}{127.0.0.1:46799}
[junit4] 2> 556653 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.e.j.s.Server Started @559546ms
[junit4] 2> 556653 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, hostContext=/ov/cn, hostPort=46799, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/shard-2-001/cores}
[junit4] 2> 556653 ERROR (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 556654 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 556654 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 556654 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 556654 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-07-15T08:38:48.215Z
[junit4] 2> 556658 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 556658 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/shard-2-001/solr.xml
[junit4] 2> 556664 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 556668 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52694/solr
[junit4] 2> 556685 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:46799_ov%2Fcn ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 556687 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:46799_ov%2Fcn ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 556690 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:46799_ov%2Fcn ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46799_ov%2Fcn
[junit4] 2> 556691 INFO (zkCallback-344-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 556692 INFO (zkCallback-349-thread-1-processing-n:127.0.0.1:32870_ov%2Fcn) [n:127.0.0.1:32870_ov%2Fcn ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 556692 INFO (zkCallback-337-thread-2-processing-n:127.0.0.1:41805_ov%2Fcn) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 556708 INFO (zkCallback-355-thread-1-processing-n:127.0.0.1:46799_ov%2Fcn) [n:127.0.0.1:46799_ov%2Fcn ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 556788 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:46799_ov%2Fcn ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
[junit4] 2> 556794 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:46799_ov%2Fcn ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
[junit4] 2> 556794 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:46799_ov%2Fcn ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
[junit4] 2> 556796 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [n:127.0.0.1:46799_ov%2Fcn ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/shard-2-001/cores
[junit4] 2> 556830 INFO (qtp556104417-4455) [n:127.0.0.1:46799_ov%2Fcn ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:46799_ov%252Fcn&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 556833 INFO (OverseerThreadFactory-758-thread-4-processing-n:127.0.0.1:41805_ov%2Fcn) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:46799_ov%2Fcn for creating new replica
[junit4] 2> 556833 INFO (OverseerCollectionConfigSetProcessor-98311072817348612-127.0.0.1:41805_ov%2Fcn-n_0000000000) [n:127.0.0.1:41805_ov%2Fcn ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 556836 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n0&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 556837 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 557871 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 557886 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.s.IndexSchema [collection1_shard1_replica_n0] Schema name=test
[junit4] 2> 558012 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 558032 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n0' using configuration from collection collection1, trusted=true
[junit4] 2> 558035 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n0' (registry 'solr.core.collection1.shard1.replica_n0') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@152e00de
[junit4] 2> 558036 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:49971/solr_hdfs_home
[junit4] 2> 558036 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 558036 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 558036 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.SolrCore [[collection1_shard1_replica_n0] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/shard-2-001/cores/collection1_shard1_replica_n0], dataDir=[hdfs://localhost:49971/solr_hdfs_home/collection1/core_node2/data/]
[junit4] 2> 558037 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49971/solr_hdfs_home/collection1/core_node2/data/snapshot_metadata
[junit4] 2> 558047 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 558047 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 558053 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 558054 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49971/solr_hdfs_home/collection1/core_node2/data
[junit4] 2> 558079 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49971/solr_hdfs_home/collection1/core_node2/data/index
[junit4] 2> 558087 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 558087 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 558093 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 558093 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=7, maxMergeAtOnceExplicit=2, maxMergedSegmentMB=54.2822265625, floorSegmentMB=1.00390625, forceMergeDeletesPctAllowed=4.833164813857602, segmentsPerTier=10.0, maxCFSSegmentSizeMB=0.6904296875, noCFSRatio=0.0
[junit4] 2> 558138 INFO (IPC Server handler 5 on 49971) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:50603 is added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-4f0f9719-f907-44eb-a680-fe795758da6b:NORMAL:127.0.0.1:60224|RBW], ReplicaUC[[DISK]DS-ada032ef-beb3-4321-b044-13278d645633:NORMAL:127.0.0.1:50603|FINALIZED]]} size 0
[junit4] 2> 558139 INFO (IPC Server handler 6 on 49971) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:60224 is added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-ada032ef-beb3-4321-b044-13278d645633:NORMAL:127.0.0.1:50603|FINALIZED], ReplicaUC[[DISK]DS-7b441025-265d-40cb-ac06-5e2d91c89e3b:NORMAL:127.0.0.1:60224|FINALIZED]]} size 0
[junit4] 2> 558145 WARN (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 558201 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 558201 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 558201 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 558230 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 558230 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 558233 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=13, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=10.5625, floorSegmentMB=0.54296875, forceMergeDeletesPctAllowed=17.113764658263925, segmentsPerTier=33.0, maxCFSSegmentSizeMB=2.0732421875, noCFSRatio=1.0
[junit4] 2> 558241 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.s.SolrIndexSearcher Opening [Searcher@6eee6400[collection1_shard1_replica_n0] main]
[junit4] 2> 558242 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 558243 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 558243 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 558246 INFO (searcherExecutor-783-thread-1-processing-n:127.0.0.1:46799_ov%2Fcn x:collection1_shard1_replica_n0 s:shard1 c:collection1) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.SolrCore [collection1_shard1_replica_n0] Registered new searcher Searcher@6eee6400[collection1_shard1_replica_n0] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 558246 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1572977172605304832
[junit4] 2> 558256 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 558256 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 558256 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:46799/ov/cn/collection1_shard1_replica_n0/
[junit4] 2> 558256 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 558256 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.SyncStrategy http://127.0.0.1:46799/ov/cn/collection1_shard1_replica_n0/ has no replicas
[junit4] 2> 558256 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 558262 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:46799/ov/cn/collection1_shard1_replica_n0/ shard1
[junit4] 2> 558413 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 558416 INFO (qtp556104417-4457) [n:127.0.0.1:46799_ov%2Fcn c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n0&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1579
[junit4] 2> 558418 INFO (qtp556104417-4455) [n:127.0.0.1:46799_ov%2Fcn ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:46799_ov%252Fcn&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=1587
[junit4] 2> 558558 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/shard-3-001 of type NRT
[junit4] 2> 558559 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 558565 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6769a868{/ov/cn,null,AVAILABLE}
[junit4] 2> 558565 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@44cc5e5d{HTTP/1.1,[http/1.1]}{127.0.0.1:45916}
[junit4] 2> 558566 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.e.j.s.Server Started @561459ms
[junit4] 2> 558566 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/ov/cn, hostPort=45916, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_793723D4943B9E3D-001/shard-3-001/cores}
[junit4] 2> 558566 ERROR (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 558567 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 558567 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 558567 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 558567 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-07-15T08:38:50.128Z
[junit4] 2> 558577 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[793723D4943B9E3D]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome
[...truncated too long message...]
2> 2781182 INFO (commitScheduler-10170-thread-1) [ x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2781184 INFO (searcherExecutor-10167-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1545d515[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_5(7.1.0):C1)))}
[junit4] 2> 2781432 INFO (TEST-AutoCommitTest.testMaxTime-seed#[793723D4943B9E3D]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[550 (1572979503784787968)]} 0 3270
[junit4] 2> 2781433 INFO (TEST-AutoCommitTest.testMaxTime-seed#[793723D4943B9E3D]) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=null path=null params={q=id:529&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
[junit4] 2> 2781433 INFO (TEST-AutoCommitTest.testMaxTime-seed#[793723D4943B9E3D]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1572979503785836544)]} 0 3271
[junit4] 2> 2781433 INFO (TEST-AutoCommitTest.testMaxTime-seed#[793723D4943B9E3D]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1572979503785836545)]} 0 3271
[junit4] 2> 2781433 INFO (TEST-AutoCommitTest.testMaxTime-seed#[793723D4943B9E3D]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1572979503785836546)]} 0 3271
[junit4] 2> 2781434 INFO (TEST-AutoCommitTest.testMaxTime-seed#[793723D4943B9E3D]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1572979503786885120)]} 0 3271
[junit4] 2> 2781434 INFO (TEST-AutoCommitTest.testMaxTime-seed#[793723D4943B9E3D]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1572979503786885121)]} 0 3271
[junit4] 2> 2781434 INFO (TEST-AutoCommitTest.testMaxTime-seed#[793723D4943B9E3D]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1572979503786885122)]} 0 3272
[junit4] 2> 2781434 INFO (TEST-AutoCommitTest.testMaxTime-seed#[793723D4943B9E3D]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1572979503786885123)]} 0 3272
[junit4] 2> 2781434 INFO (TEST-AutoCommitTest.testMaxTime-seed#[793723D4943B9E3D]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1572979503786885124)]} 0 3272
[junit4] 2> 2781434 INFO (TEST-AutoCommitTest.testMaxTime-seed#[793723D4943B9E3D]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1572979503786885125)]} 0 3272
[junit4] 2> 2781435 INFO (TEST-AutoCommitTest.testMaxTime-seed#[793723D4943B9E3D]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1572979503787933696)]} 0 3272
[junit4] 2> 2781435 INFO (TEST-AutoCommitTest.testMaxTime-seed#[793723D4943B9E3D]) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=null path=null params={q=id:500&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
[junit4] 2> 2782933 INFO (commitScheduler-10170-thread-1) [ x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
[junit4] 2> 2782938 INFO (commitScheduler-10170-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3ef9c717[collection1] main]
[junit4] 2> 2782938 INFO (commitScheduler-10170-thread-1) [ x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2782940 INFO (searcherExecutor-10167-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3ef9c717[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_5(7.1.0):C1) Uninverting(_6(7.1.0):C11/9:delGen=1)))}
[junit4] 2> 2783186 INFO (TEST-AutoCommitTest.testMaxTime-seed#[793723D4943B9E3D]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[531 (1572979505623990272)]} 0 5024
[junit4] 2> 2783187 INFO (TEST-AutoCommitTest.testMaxTime-seed#[793723D4943B9E3D]) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=null path=null params={q=id:500&qt=&start=0&rows=20&version=2.2} hits=1 status=0 QTime=0
[junit4] 2> 2783187 INFO (TEST-AutoCommitTest.testMaxTime-seed#[793723D4943B9E3D]) [ x:collection1] o.a.s.SolrTestCaseJ4 ###Ending testMaxTime
[junit4] 2> 2783188 INFO (SUITE-AutoCommitTest-seed#[793723D4943B9E3D]-worker) [ x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> 2783188 INFO (SUITE-AutoCommitTest-seed#[793723D4943B9E3D]-worker) [ x:collection1] o.a.s.c.CoreContainer Shutting down CoreContainer instance=766467968
[junit4] 2> 2783188 INFO (SUITE-AutoCommitTest-seed#[793723D4943B9E3D]-worker) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 2783189 INFO (SUITE-AutoCommitTest-seed#[793723D4943B9E3D]-worker) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 2783190 INFO (SUITE-AutoCommitTest-seed#[793723D4943B9E3D]-worker) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 2783191 INFO (SUITE-AutoCommitTest-seed#[793723D4943B9E3D]-worker) [ x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2783192 INFO (coreCloseExecutor-10171-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@282d6691
[junit4] 2> 2783192 INFO (coreCloseExecutor-10171-thread-1-processing-x:collection1) [ x:collection1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter close.
[junit4] 2> 2783192 INFO (coreCloseExecutor-10171-thread-1-processing-x:collection1) [ x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@42fead80 commitCommandVersion:0
[junit4] 2> 2783195 INFO (coreCloseExecutor-10171-thread-1-processing-x:collection1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=674064017
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.AutoCommitTest_793723D4943B9E3D-001
[junit4] 2> NOTE: test params are: codec=Lucene70, sim=RandomSimilarity(queryNorm=false): {}, locale=zh, timezone=Europe/Vienna
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_131 (64-bit)/cpus=4,threads=1,free=311877216,total=520093696
[junit4] 2> NOTE: All tests run in this JVM: [FieldAnalysisRequestHandlerTest, BitVectorTest, TestBlobHandler, TestConfigSetProperties, TestFieldResource, CreateCollectionCleanupTest, TestRandomCollapseQParserPlugin, TestRandomFaceting, TestTolerantUpdateProcessorRandomCloud, TestCustomStream, TestSimpleQParserPlugin, TestCloudRecovery, TestReplicationHandlerBackup, TestPHPSerializedResponseWriter, HdfsBasicDistributedZk2Test, TestMaxTokenLenTokenizer, HttpPartitionTest, SharedFSAutoReplicaFailoverTest, ExitableDirectoryReaderTest, MBeansHandlerTest, LeaderElectionIntegrationTest, MoveReplicaHDFSUlogDirTest, TestPhraseSuggestions, TestLegacyFieldReuse, TestReplicaProperties, DeleteShardTest, TestLegacyNumericRangeQueryBuilder, TestLMDirichletSimilarityFactory, OverseerStatusTest, BlobRepositoryCloudTest, TestSSLRandomization, SolrRequestParserTest, WordBreakSolrSpellCheckerTest, TestNestedDocsSort, IgnoreCommitOptimizeUpdateProcessorFactoryTest, DistributedSuggestComponentTest, CdcrRequestHandlerTest, ChaosMonkeyNothingIsSafeTest, TestSQLHandler, TestCustomSort, ChaosMonkeySafeLeaderWithPullReplicasTest, OverseerCollectionConfigSetProcessorTest, TestRawTransformer, TestRandomDVFaceting, TestInfoStreamLogging, TestJsonRequest, TestClassicSimilarityFactory, TestLeaderElectionWithEmptyReplica, TestAuthenticationFramework, TestCollapseQParserPlugin, TestBinaryField, TestSolrCoreSnapshots, SolrIndexSplitterTest, SolrCoreTest, ImplicitSnitchTest, DistributedVersionInfoTest, DistributedFacetPivotLongTailTest, TestImpersonationWithHadoopAuth, TestInPlaceUpdatesDistrib, DebugComponentTest, TestConfig, BlockDirectoryTest, AtomicUpdatesTest, TestSolr4Spatial, DeleteLastCustomShardedReplicaTest, DocumentAnalysisRequestHandlerTest, TestFunctionQuery, TestCryptoKeys, MetricsConfigTest, TestConfigsApi, AssignTest, TestExpandComponent, ConnectionReuseTest, TestAnalyzeInfixSuggestions, TestConfigSetsAPI, BlockJoinFacetSimpleTest, ConfigureRecoveryStrategyTest, DistribDocExpirationUpdateProcessorTest, TestMultiValuedNumericRangeQuery, TestRuleBasedAuthorizationPlugin, TestCloudManagedSchema, TestRangeQuery, SmileWriterTest, BJQParserTest, TestBulkSchemaAPI, TestDocSet, TestStressLiveNodes, TestLuceneMatchVersion, TestDownShardTolerantSearch, TestBackupRepositoryFactory, TestDocBasedVersionConstraints, TestRandomFlRTGCloud, TestFieldCacheVsDocValues, TestDynamicFieldCollectionResource, ResponseHeaderTest, TestExtendedDismaxParser, UUIDFieldTest, SolrCloudReportersTest, MetricUtilsTest, TestReloadAndDeleteDocs, HdfsUnloadDistributedZkTest, TestJavabinTupleStreamParser, OpenExchangeRatesOrgProviderTest, BasicZkTest, TestSolrCloudWithSecureImpersonation, AddBlockUpdateTest, StatsReloadRaceTest, TestSolrCoreParser, TestPayloadCheckQParserPlugin, TestTrackingShardHandlerFactory, TestCloudJSONFacetJoinDomain, TestMultiWordSynonyms, DistributedFacetPivotWhiteBoxTest, HighlighterConfigTest, EchoParamsTest, TestBadConfig, MigrateRouteKeyTest, AutoCommitTest]
[junit4] Completed [376/728 (2!)] on J0 in 15.44s, 3 tests, 1 error <<< FAILURES!
[...truncated 44468 lines...]
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 9 - Failure
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/9/
6 tests failed.
FAILED: org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest.test
Error Message:
The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
Stack Trace:
java.lang.AssertionError: The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
at __randomizedtesting.SeedInfo.seed([ABCE03BB53DB72B5:239A3C61FD271F4D]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.ChaosMonkey.stopTheMonkey(ChaosMonkey.java:587)
at org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest.test(ChaosMonkeySafeLeaderWithPullReplicasTest.java:174)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest
Error Message:
6 threads leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest: 1) Thread[id=90000, name=StoppableCommitThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.StoppableCommitThread.run(StoppableCommitThread.java:55) 2) Thread[id=90037, name=StoppableCommitThread-EventThread, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501) 3) Thread[id=90036, name=StoppableCommitThread-SendThread(127.0.0.1:37765), state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1051) 4) Thread[id=90038, name=zkCallback-3567-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=90087, name=zkCallback-3567-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=90086, name=zkCallback-3567-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 6 threads leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest:
1) Thread[id=90000, name=StoppableCommitThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.StoppableCommitThread.run(StoppableCommitThread.java:55)
2) Thread[id=90037, name=StoppableCommitThread-EventThread, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)
3) Thread[id=90036, name=StoppableCommitThread-SendThread(127.0.0.1:37765), state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1051)
4) Thread[id=90038, name=zkCallback-3567-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=90087, name=zkCallback-3567-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
6) Thread[id=90086, name=zkCallback-3567-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([ABCE03BB53DB72B5]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=90036, name=StoppableCommitThread-SendThread(127.0.0.1:37765), state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060) 2) Thread[id=90038, name=zkCallback-3567-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=90087, name=zkCallback-3567-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=90086, name=zkCallback-3567-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=90036, name=StoppableCommitThread-SendThread(127.0.0.1:37765), state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
2) Thread[id=90038, name=zkCallback-3567-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=90087, name=zkCallback-3567-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=90086, name=zkCallback-3567-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([ABCE03BB53DB72B5]:0)
FAILED: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test
Error Message:
Java heap space
Stack Trace:
java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([ABCE03BB53DB72B5:239A3C61FD271F4D]:0)
at java.util.Arrays.copyOf(Arrays.java:3332)
at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124)
at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:649)
at java.lang.StringBuilder.append(StringBuilder.java:202)
at org.apache.http.client.utils.URLEncodedUtils.urlEncode(URLEncodedUtils.java:522)
at org.apache.http.client.utils.URLEncodedUtils.encodeFormFields(URLEncodedUtils.java:638)
at org.apache.http.client.utils.URLEncodedUtils.format(URLEncodedUtils.java:390)
at org.apache.http.client.utils.URLEncodedUtils.format(URLEncodedUtils.java:369)
at org.apache.http.client.entity.UrlEncodedFormEntity.<init>(UrlEncodedFormEntity.java:77)
at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:447)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:250)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1121)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:974)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:990)
at org.apache.solr.cloud.CloudInspectUtil.compareResults(CloudInspectUtil.java:228)
at org.apache.solr.cloud.CloudInspectUtil.compareResults(CloudInspectUtil.java:167)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testIndexingBatchPerRequestWithHttpSolrClient(FullSolrCloudDistribCmdsTest.java:668)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test(FullSolrCloudDistribCmdsTest.java:152)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
FAILED: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test
Error Message:
Timeout waiting for all live and active
Stack Trace:
java.lang.AssertionError: Timeout waiting for all live and active
at __randomizedtesting.SeedInfo.seed([ABCE03BB53DB72B5:239A3C61FD271F4D]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:246)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:136)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload
Error Message:
expected:<[{indexVersion=1500018646551,generation=2,filelist=[_w3.cfe, _w3.cfs, _w3.si, _x5.fdt, _x5.fdx, _x5.fnm, _x5.nvd, _x5.nvm, _x5.si, _x5_Memory_0.ram, _x9.fdt, _x9.fdx, _x9.fnm, _x9.nvd, _x9.nvm, _x9.si, _x9_Memory_0.ram, _xb.cfe, _xb.cfs, _xb.si, _xc.fdt, _xc.fdx, _xc.fnm, _xc.nvd, _xc.nvm, _xc.si, _xc_Memory_0.ram, _xd.fdt, _xd.fdx, _xd.fnm, _xd.nvd, _xd.nvm, _xd.si, _xd_Memory_0.ram, _xe.fdt, _xe.fdx, _xe.fnm, _xe.nvd, _xe.nvm, _xe.si, _xe_Memory_0.ram, segments_2]}]> but was:<[{indexVersion=1500018646551,generation=2,filelist=[_w3.cfe, _w3.cfs, _w3.si, _x5.fdt, _x5.fdx, _x5.fnm, _x5.nvd, _x5.nvm, _x5.si, _x5_Memory_0.ram, _x9.fdt, _x9.fdx, _x9.fnm, _x9.nvd, _x9.nvm, _x9.si, _x9_Memory_0.ram, _xb.cfe, _xb.cfs, _xb.si, _xc.fdt, _xc.fdx, _xc.fnm, _xc.nvd, _xc.nvm, _xc.si, _xc_Memory_0.ram, _xd.fdt, _xd.fdx, _xd.fnm, _xd.nvd, _xd.nvm, _xd.si, _xd_Memory_0.ram, _xe.fdt, _xe.fdx, _xe.fnm, _xe.nvd, _xe.nvm, _xe.si, _xe_Memory_0.ram, segments_2]}, {indexVersion=1500018646551,generation=3,filelist=[_x4.cfe, _x4.cfs, _x4.si, _x5.fdt, _x5.fdx, _x5.fnm, _x5.nvd, _x5.nvm, _x5.si, _x5_Memory_0.ram, _x9.fdt, _x9.fdx, _x9.fnm, _x9.nvd, _x9.nvm, _x9.si, _x9_Memory_0.ram, _xb.cfe, _xb.cfs, _xb.si, _xc.fdt, _xc.fdx, _xc.fnm, _xc.nvd, _xc.nvm, _xc.si, _xc_Memory_0.ram, _xd.fdt, _xd.fdx, _xd.fnm, _xd.nvd, _xd.nvm, _xd.si, _xd_Memory_0.ram, _xe.fdt, _xe.fdx, _xe.fnm, _xe.nvd, _xe.nvm, _xe.si, _xe_Memory_0.ram, segments_3]}]>
Stack Trace:
java.lang.AssertionError: expected:<[{indexVersion=1500018646551,generation=2,filelist=[_w3.cfe, _w3.cfs, _w3.si, _x5.fdt, _x5.fdx, _x5.fnm, _x5.nvd, _x5.nvm, _x5.si, _x5_Memory_0.ram, _x9.fdt, _x9.fdx, _x9.fnm, _x9.nvd, _x9.nvm, _x9.si, _x9_Memory_0.ram, _xb.cfe, _xb.cfs, _xb.si, _xc.fdt, _xc.fdx, _xc.fnm, _xc.nvd, _xc.nvm, _xc.si, _xc_Memory_0.ram, _xd.fdt, _xd.fdx, _xd.fnm, _xd.nvd, _xd.nvm, _xd.si, _xd_Memory_0.ram, _xe.fdt, _xe.fdx, _xe.fnm, _xe.nvd, _xe.nvm, _xe.si, _xe_Memory_0.ram, segments_2]}]> but was:<[{indexVersion=1500018646551,generation=2,filelist=[_w3.cfe, _w3.cfs, _w3.si, _x5.fdt, _x5.fdx, _x5.fnm, _x5.nvd, _x5.nvm, _x5.si, _x5_Memory_0.ram, _x9.fdt, _x9.fdx, _x9.fnm, _x9.nvd, _x9.nvm, _x9.si, _x9_Memory_0.ram, _xb.cfe, _xb.cfs, _xb.si, _xc.fdt, _xc.fdx, _xc.fnm, _xc.nvd, _xc.nvm, _xc.si, _xc_Memory_0.ram, _xd.fdt, _xd.fdx, _xd.fnm, _xd.nvd, _xd.nvm, _xd.si, _xd_Memory_0.ram, _xe.fdt, _xe.fdx, _xe.fnm, _xe.nvd, _xe.nvm, _xe.si, _xe_Memory_0.ram, segments_2]}, {indexVersion=1500018646551,generation=3,filelist=[_x4.cfe, _x4.cfs, _x4.si, _x5.fdt, _x5.fdx, _x5.fnm, _x5.nvd, _x5.nvm, _x5.si, _x5_Memory_0.ram, _x9.fdt, _x9.fdx, _x9.fnm, _x9.nvd, _x9.nvm, _x9.si, _x9_Memory_0.ram, _xb.cfe, _xb.cfs, _xb.si, _xc.fdt, _xc.fdx, _xc.fnm, _xc.nvd, _xc.nvm, _xc.si, _xc_Memory_0.ram, _xd.fdt, _xd.fdx, _xd.fnm, _xd.nvd, _xd.nvm, _xd.si, _xd_Memory_0.ram, _xe.fdt, _xe.fdx, _xe.fnm, _xe.nvd, _xe.nvm, _xe.si, _xe_Memory_0.ram, segments_3]}]>
at __randomizedtesting.SeedInfo.seed([ABCE03BB53DB72B5:8E19188B23937CB6]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:147)
at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1277)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12426 lines...]
[junit4] Suite: org.apache.solr.handler.TestReplicationHandler
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABCE03BB53DB72B5-001/init-core-data-001
[junit4] 2> 2896291 WARN (SUITE-TestReplicationHandler-seed#[ABCE03BB53DB72B5]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
[junit4] 2> 2896292 INFO (SUITE-TestReplicationHandler-seed#[ABCE03BB53DB72B5]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 2896293 INFO (SUITE-TestReplicationHandler-seed#[ABCE03BB53DB72B5]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
[junit4] 2> 2896294 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.SolrTestCaseJ4 ###Starting doTestRepeater
[junit4] 2> 2896294 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABCE03BB53DB72B5-001/solr-instance-001/collection1
[junit4] 2> 2896296 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2896297 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2306f184{/solr,null,AVAILABLE}
[junit4] 2> 2896297 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1065cb39{HTTP/1.1,[http/1.1]}{127.0.0.1:41611}
[junit4] 2> 2896298 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.e.j.s.Server Started @2905864ms
[junit4] 2> 2896298 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABCE03BB53DB72B5-001/solr-instance-001/collection1/data, hostContext=/solr, hostPort=41611}
[junit4] 2> 2896298 ERROR (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2896298 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 2896298 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 2896298 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 2896298 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-07-14T07:46:53.862Z
[junit4] 2> 2896298 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABCE03BB53DB72B5-001/solr-instance-001/solr.xml
[junit4] 2> 2896302 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@68f7814b, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 2896337 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@68f7814b
[junit4] 2> 2896340 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@68f7814b
[junit4] 2> 2896340 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@68f7814b
[junit4] 2> 2896341 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABCE03BB53DB72B5-001/solr-instance-001/.
[junit4] 2> 2896341 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2896350 INFO (coreLoadExecutor-7789-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 2896386 INFO (coreLoadExecutor-7789-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2896396 INFO (coreLoadExecutor-7789-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 2896396 INFO (coreLoadExecutor-7789-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABCE03BB53DB72B5-001/solr-instance-001/./collection1, trusted=true
[junit4] 2> 2896396 INFO (coreLoadExecutor-7789-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@68f7814b
[junit4] 2> 2896396 INFO (coreLoadExecutor-7789-thread-1) [ x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2896396 INFO (coreLoadExecutor-7789-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABCE03BB53DB72B5-001/solr-instance-001/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABCE03BB53DB72B5-001/solr-instance-001/./collection1/data/]
[junit4] 2> 2896397 INFO (coreLoadExecutor-7789-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=45, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6226604577629139]
[junit4] 2> 2896479 INFO (coreLoadExecutor-7789-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2896479 INFO (coreLoadExecutor-7789-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2896480 INFO (coreLoadExecutor-7789-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@188a3d90
[junit4] 2> 2896480 INFO (coreLoadExecutor-7789-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@61b1c677[collection1] main]
[junit4] 2> 2896480 INFO (coreLoadExecutor-7789-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABCE03BB53DB72B5-001/solr-instance-001/collection1/conf
[junit4] 2> 2896489 INFO (coreLoadExecutor-7789-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml,xslt/dummy.xsl
[junit4] 2> 2896489 INFO (coreLoadExecutor-7789-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2896490 INFO (searcherExecutor-7790-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@61b1c677[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2896886 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABCE03BB53DB72B5-001/solr-instance-002/collection1
[junit4] 2> 2896888 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2896889 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@79653f7e{/solr,null,AVAILABLE}
[junit4] 2> 2896889 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5f8fe966{HTTP/1.1,[http/1.1]}{127.0.0.1:41753}
[junit4] 2> 2896889 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.e.j.s.Server Started @2906456ms
[junit4] 2> 2896889 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABCE03BB53DB72B5-001/solr-instance-002/collection1/data, hostContext=/solr, hostPort=41753}
[junit4] 2> 2896890 ERROR (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2896890 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 2896890 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 2896890 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 2896890 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-07-14T07:46:54.454Z
[junit4] 2> 2896891 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABCE03BB53DB72B5-001/solr-instance-002/solr.xml
[junit4] 2> 2896940 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@68f7814b, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 2896982 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@68f7814b
[junit4] 2> 2896986 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@68f7814b
[junit4] 2> 2896986 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@68f7814b
[junit4] 2> 2896988 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABCE03BB53DB72B5-001/solr-instance-002/.
[junit4] 2> 2896988 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2896996 INFO (coreLoadExecutor-7799-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 2897028 INFO (coreLoadExecutor-7799-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2897029 INFO (coreLoadExecutor-7799-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 2897030 INFO (coreLoadExecutor-7799-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABCE03BB53DB72B5-001/solr-instance-002/./collection1, trusted=true
[junit4] 2> 2897030 INFO (coreLoadExecutor-7799-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@68f7814b
[junit4] 2> 2897030 INFO (coreLoadExecutor-7799-thread-1) [ x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2897030 INFO (coreLoadExecutor-7799-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABCE03BB53DB72B5-001/solr-instance-002/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABCE03BB53DB72B5-001/solr-instance-002/./collection1/data/]
[junit4] 2> 2897031 INFO (coreLoadExecutor-7799-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=45, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6226604577629139]
[junit4] 2> 2897093 INFO (coreLoadExecutor-7799-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2897093 INFO (coreLoadExecutor-7799-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2897094 INFO (coreLoadExecutor-7799-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@1e2a6d93
[junit4] 2> 2897094 INFO (coreLoadExecutor-7799-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3c895466[collection1] main]
[junit4] 2> 2897094 INFO (coreLoadExecutor-7799-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABCE03BB53DB72B5-001/solr-instance-002/collection1/conf
[junit4] 2> 2897099 INFO (coreLoadExecutor-7799-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms
[junit4] 2> 2897100 INFO (searcherExecutor-7800-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3c895466[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2897100 INFO (qtp1630951948-121935) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2897100 INFO (indexFetcher-7804-thread-1) [ x:collection1] o.a.s.h.IndexFetcher Master's generation: 1
[junit4] 2> 2897101 INFO (indexFetcher-7804-thread-1) [ x:collection1] o.a.s.h.IndexFetcher Master's version: 0
[junit4] 2> 2897101 INFO (indexFetcher-7804-thread-1) [ x:collection1] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 2897101 INFO (indexFetcher-7804-thread-1) [ x:collection1] o.a.s.h.IndexFetcher Slave's version: 0
[junit4] 2> 2897554 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@5f8fe966{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 2897554 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1918352296
[junit4] 2> 2897554 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 2897555 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 2897555 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 2897555 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 2897556 INFO (coreCloseExecutor-7805-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@658fb64f
[junit4] 2> 2897557 INFO (coreCloseExecutor-7805-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=1703917135
[junit4] 2> 2897558 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@79653f7e{/solr,null,UNAVAILABLE}
[junit4] 2> 2897559 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2897559 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@78e741bd{/solr,null,AVAILABLE}
[junit4] 2> 2897560 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4a0127ff{HTTP/1.1,[http/1.1]}{127.0.0.1:35054}
[junit4] 2> 2897560 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.e.j.s.Server Started @2907126ms
[junit4] 2> 2897560 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABCE03BB53DB72B5-001/solr-instance-002/collection1/data, hostContext=/solr, hostPort=35054}
[junit4] 2> 2897560 ERROR (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2897560 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 2897560 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 2897560 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 2897560 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-07-14T07:46:55.124Z
[junit4] 2> 2897561 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABCE03BB53DB72B5-001/solr-instance-002/solr.xml
[junit4] 2> 2897565 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@68f7814b, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 2897600 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@68f7814b
[junit4] 2> 2897604 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@68f7814b
[junit4] 2> 2897604 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@68f7814b
[junit4] 2> 2897605 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABCE03BB53DB72B5-001/solr-instance-002/.
[junit4] 2> 2897605 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2897634 INFO (coreLoadExecutor-7812-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 2897670 INFO (coreLoadExecutor-7812-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2897671 INFO (coreLoadExecutor-7812-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 2897671 INFO (coreLoadExecutor-7812-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABCE03BB53DB72B5-001/solr-instance-002/./collection1, trusted=true
[junit4] 2> 2897672 INFO (coreLoadExecutor-7812-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@68f7814b
[junit4] 2> 2897672 INFO (coreLoadExecutor-7812-thread-1) [ x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2897684 INFO (coreLoadExecutor-7812-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABCE03BB53DB72B5-001/solr-instance-002/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABCE03BB53DB72B5-001/solr-instance-002/./collection1/data/]
[junit4] 2> 2897685 INFO (coreLoadExecutor-7812-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=45, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6226604577629139]
[junit4] 2> 2897814 INFO (coreLoadExecutor-7812-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2897814 INFO (coreLoadExecutor-7812-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2897827 INFO (coreLoadExecutor-7812-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@6deb6047
[junit4] 2> 2897827 INFO (coreLoadExecutor-7812-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@f8f0809[collection1] main]
[junit4] 2> 2897827 INFO (coreLoadExecutor-7812-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABCE03BB53DB72B5-001/solr-instance-002/collection1/conf
[junit4] 2> 2897828 INFO (coreLoadExecutor-7812-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2897828 INFO (searcherExecutor-7813-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@f8f0809[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2898157 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABCE03BB53DB72B5-001/solr-instance-003/collection1
[junit4] 2> 2898159 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2898194 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@227ab761{/solr,null,AVAILABLE}
[junit4] 2> 2898248 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3e8329f0{HTTP/1.1,[http/1.1]}{127.0.0.1:44104}
[junit4] 2> 2898248 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.e.j.s.Server Started @2907815ms
[junit4] 2> 2898248 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABCE03BB53DB72B5-001/solr-instance-003/collection1/data, hostContext=/solr, hostPort=44104}
[junit4] 2> 2898248 ERROR (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2898249 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 2898249 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 2898249 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 2898249 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-07-14T07:46:55.813Z
[junit4] 2> 2898249 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABCE03BB53DB72B5-001/solr-instance-003/solr.xml
[junit4] 2> 2898253 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@68f7814b, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 2898313 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@68f7814b
[junit4] 2> 2898317 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@68f7814b
[junit4] 2> 2898317 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@68f7814b
[junit4] 2> 2898323 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABCE03BB53DB72B5-001/solr-instance-003/.
[junit4] 2> 2898323 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2898333 INFO (coreLoadExecutor-7822-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 2898374 INFO (coreLoadExecutor-7822-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2898392 INFO (coreLoadExecutor-7822-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 2898392 INFO (coreLoadExecutor-7822-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABCE03BB53DB72B5-001/solr-instance-003/./collection1, trusted=true
[junit4] 2> 2898392 INFO (coreLoadExecutor-7822-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@68f7814b
[junit4] 2> 2898392 INFO (coreLoadExecutor-7822-thread-1) [ x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2898392 INFO (coreLoadExecutor-7822-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABCE03BB53DB72B5-001/solr-instance-003/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABCE03BB53DB72B5-001/solr-instance-003/./collection1/data/]
[junit4] 2> 2898393 INFO (coreLoadExecutor-7822-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=45, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6226604577629139]
[junit4] 2> 2898435 INFO (coreLoadExecutor-7822-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2898435 INFO (coreLoadExecutor-7822-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2898436 INFO (coreLoadExecutor-7822-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@2d34a4f9
[junit4] 2> 2898436 INFO (coreLoadExecutor-7822-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@58c5d3c0[collection1] main]
[junit4] 2> 2898436 INFO (coreLoadExecutor-7822-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABCE03BB53DB72B5-001/solr-instance-003/collection1/conf
[junit4] 2> 2898437 WARN (coreLoadExecutor-7822-thread-1) [ x:collection1] o.a.s.h.IndexFetcher 'masterUrl' must be specified without the /replication suffix
[junit4] 2> 2898439 INFO (coreLoadExecutor-7822-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler No value set for 'pollInterval'. Timer Task not started.
[junit4] 2> 2898439 INFO (coreLoadExecutor-7822-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml
[junit4] 2> 2898439 INFO (coreLoadExecutor-7822-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2898439 INFO (searcherExecutor-7823-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@58c5d3c0[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2898904 INFO (qtp1630951948-121939) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[0]} 0 0
[junit4] 2> 2898904 INFO (qtp1630951948-121932) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1]} 0 0
[junit4] 2> 2898905 INFO (qtp1630951948-121935) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[2]} 0 0
[junit4] 2> 2898905 INFO (qtp1630951948-121936) [ x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2898905 INFO (qtp1630951948-121936) [ x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@158f28cf commitCommandVersion:0
[junit4] 2> 2898907 INFO (qtp1630951948-121936) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@31b2070a[collection1] main]
[junit4] 2> 2898907 INFO (qtp1630951948-121936) [ x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2898908 INFO (searcherExecutor-7790-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@31b2070a[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.1.0):C3)))}
[junit4] 2> 2898908 INFO (qtp1630951948-121936) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 3
[junit4] 2> 2898910 WARN (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher 'masterUrl' must be specified without the /replication suffix
[junit4] 2> 2898911 INFO (qtp1630951948-121933) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2898911 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Master's generation: 2
[junit4] 2> 2898911 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Master's version: 1500018416469
[junit4] 2> 2898911 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 2898911 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Slave's version: 0
[junit4] 2> 2898911 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Starting replication process
[junit4] 2> 2898913 INFO (qtp1630951948-121939) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&wt=javabin&version=2&command=filelist} status=0 QTime=1
[junit4] 2> 2898913 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Number of files in latest index in master: 8
[junit4] 2> 2898914 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=38, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 2898915 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 2898915 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Starting download (fullCopy=false) to MockDirectoryWrapper(RAMDirectory@742ac711 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@349b16f4)
[junit4] 2> 2898915 INFO (qtp1630951948-121932) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.si&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2898932 INFO (qtp1630951948-121935) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.nvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2898933 INFO (qtp1630951948-121936) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdx&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2898933 INFO (qtp1630951948-121937) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Memory_0.ram&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2898934 INFO (qtp1630951948-121933) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdt&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2898934 INFO (qtp1630951948-121939) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fnm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2898936 INFO (qtp1630951948-121932) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.nvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2898937 INFO (qtp1630951948-121935) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=segments_2&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2898937 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Total time taken for download (fullCopy=false,bytesDownloaded=1380) : 0 secs (null bytes/sec) to MockDirectoryWrapper(RAMDirectory@742ac711 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@349b16f4)
[junit4] 2> 2898940 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=41, maxMergeAtOnceExplicit=30, maxMergedSegmentMB=1.57421875, floorSegmentMB=1.9482421875, forceMergeDeletesPctAllowed=8.568864096386566, segmentsPerTier=20.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.39460049999639224
[junit4] 2> 2898941 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 2898941 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@750ed32f[collection1] main]
[junit4] 2> 2898942 INFO (searcherExecutor-7823-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@750ed32f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.1.0):C3)))}
[junit4] 2> 2898942 INFO (qtp55242568-121991) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={wait=true&masterUrl=http://127.0.0.1:41611/solr/collection1/replication&command=fetchindex} status=0 QTime=33
[junit4] 2> 2898944 INFO (qtp55242568-121993) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/select params={q=*:*&sort=id+desc&wt=javabin&version=2} hits=3 status=0 QTime=0
[junit4] 2> 2898944 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.h.TestReplicationHandler Waited for 0ms and found 3 docs
[junit4] 2> 2898945 WARN (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher 'masterUrl' must be specified without the /replication suffix
[junit4] 2> 2898946 INFO (qtp55242568-121995) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2898946 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Master's generation: 2
[junit4] 2> 2898946 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Master's version: 1500018416469
[junit4] 2> 2898946 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 2898946 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Slave's version: 0
[junit4] 2> 2898946 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Starting replication process
[junit4] 2> 2898946 INFO (qtp55242568-121988) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&wt=javabin&version=2&command=filelist} status=0 QTime=0
[junit4] 2> 2898947 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Number of files in latest index in master: 8
[junit4] 2> 2898948 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=38, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 2898948 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 2898948 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Starting download (fullCopy=false) to MockDirectoryWrapper(RAMDirectory@7c783059 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@171da3ce)
[junit4] 2> 2898949 INFO (qtp55242568-121991) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.si&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2898949 INFO (qtp55242568-121989) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.nvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2898950 INFO (qtp55242568-121993) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdx&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2898951 INFO (qtp55242568-121994) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Memory_0.ram&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2898951 INFO (qtp55242568-121995) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdt&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2898960 INFO (qtp55242568-121988) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fnm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2898961 INFO (qtp55242568-121991) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.nvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2898961 INFO (qtp55242568-121989) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=segments_2&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2898962 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Total time taken for download (fullCopy=false,bytesDownloaded=1380) : 0 secs (null bytes/sec) to MockDirectoryWrapper(RAMDirectory@7c783059 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@171da3ce)
[junit4] 2> 2898964 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=41, maxMergeAtOnceExplicit=30, maxMergedSegmentMB=1.57421875, floorSegmentMB=1.9482421875, forceMergeDeletesPctAllowed=8.568864096386566, segmentsPerTier=20.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.39460049999639224
[junit4] 2> 2898965 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 2898965 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@68dff132[collection1] main]
[junit4] 2> 2898966 INFO (searcherExecutor-7813-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@68dff132[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.1.0):C3)))}
[junit4] 2> 2898966 INFO (qtp343049728-121974) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={wait=true&masterUrl=http://127.0.0.1:44104/solr/collection1/replication&command=fetchindex} status=0 QTime=21
[junit4] 2> 2898973 INFO (qtp343049728-121976) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/select params={q=*:*&sort=id+desc&wt=javabin&version=2} hits=3 status=0 QTime=0
[junit4] 2> 2898973 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[ABCE03BB53DB72B5]) [ ] o.a.s.h.TestReplicationHandler Waited for 0ms and found 3 docs
[junit4] 2> 2898973 INFO (qtp1630951948-121938) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={_trace=getDetails&qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2898974 INFO (qtp1630951948-121937) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={_trace=getDetails&qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2898975 WARN (qtp55242568-121993) [ x:collection1] o.a.s.h.ReplicationHandler Exception while invoking 'details' method for replication on master
[junit4] 2> org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://127.0.0.1:TEST_PORT/solr
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:633)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:252)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
[junit4] 2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
[junit4] 2> at org.apache.solr.handler.IndexFetcher.getDetails(IndexFetcher.java:1813)
[junit4] 2> at org.apache.solr.handler.ReplicationHandler.getReplicationDetails(ReplicationHandler.java:952)
[junit4] 2> at org.apache.solr.handler.ReplicationHandler.handleRequestBody(ReplicationHandler.java:330)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
[junit4] 2> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2473)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:720)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:526)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:378)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:322)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:462)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: org.apache.http.conn.HttpHostConnectException: Connect to 127.0.0.1:80 [/127.0.0.1] failed: Connection refused (Connection refused)
[junit4] 2> at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151)
[junit4] 2> at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
[junit4] 2> at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
[junit4] 2> at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
[junit4] 2> at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
[junit4] 2> at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
[junit4] 2> at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
[junit4] 2> at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
[junit4] 2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
[junit4] 2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:523)
[junit4] 2> ... 36 more
[junit4] 2> Caused by: java.net.ConnectException: Connection refused (Connection refused)
[junit4] 2> at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4] 2> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
[junit4] 2> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
[junit4] 2> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketIm
[...truncated too long message...]
cale=und -Dtests.timezone=Asia/Dubai -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.00s J2 | ChaosMonkeySafeLeaderWithPullReplicasTest (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 6 threads leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest:
[junit4] > 1) Thread[id=90000, name=StoppableCommitThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.solr.cloud.StoppableCommitThread.run(StoppableCommitThread.java:55)
[junit4] > 2) Thread[id=90037, name=StoppableCommitThread-EventThread, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)
[junit4] > 3) Thread[id=90036, name=StoppableCommitThread-SendThread(127.0.0.1:37765), state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1051)
[junit4] > 4) Thread[id=90038, name=zkCallback-3567-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 5) Thread[id=90087, name=zkCallback-3567-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 6) Thread[id=90086, name=zkCallback-3567-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([ABCE03BB53DB72B5]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=90036, name=StoppableCommitThread-SendThread(127.0.0.1:37765), state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
[junit4] > 2) Thread[id=90038, name=zkCallback-3567-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 3) Thread[id=90087, name=zkCallback-3567-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 4) Thread[id=90086, name=zkCallback-3567-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([ABCE03BB53DB72B5]:0)
[junit4] Completed [502/728 (4!)] on J2 in 136.56s, 1 test, 1 failure, 2 errors <<< FAILURES!
[...truncated 685 lines...]
[junit4] JVM J0: stdout was not empty, see: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/temp/junit4-J0-20170714_065827_8894968092063555578472.sysout
[junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
[junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
[junit4] Dumping heap to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/heapdumps/java_pid20311.hprof ...
[junit4] Heap dump file created [555493137 bytes in 8.784 secs]
[junit4] <<< JVM J0: EOF ----
[...truncated 7504 lines...]
BUILD FAILED
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/build.xml:818: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/build.xml:762: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid20311.hprof
Total time: 467 minutes 15 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 8 - Still unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/8/
5 tests failed.
FAILED: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test
Error Message:
The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
Stack Trace:
java.lang.AssertionError: The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
at __randomizedtesting.SeedInfo.seed([57BDCF540E9917CD:DFE9F08EA0657A35]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.ChaosMonkey.stopTheMonkey(ChaosMonkey.java:587)
at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test(ChaosMonkeySafeLeaderTest.java:133)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.update.AutoCommitTest.testCommitWithin
Error Message:
Exception during query
Stack Trace:
java.lang.RuntimeException: Exception during query
at __randomizedtesting.SeedInfo.seed([57BDCF540E9917CD:ED6FA02C8DB7F9D8]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:878)
at org.apache.solr.update.AutoCommitTest.testCommitWithin(AutoCommitTest.java:353)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
</response>
request was:q=id:529&qt=&start=0&rows=20&version=2.2
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:871)
... 40 more
FAILED: org.apache.solr.update.AutoCommitTest.testMaxTime
Error Message:
Exception during query
Stack Trace:
java.lang.RuntimeException: Exception during query
at __randomizedtesting.SeedInfo.seed([57BDCF540E9917CD:CD49B2B690038BF1]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:878)
at org.apache.solr.update.AutoCommitTest.testMaxTime(AutoCommitTest.java:270)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
</response>
request was:q=id:529&qt=&start=0&rows=20&version=2.2
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:871)
... 40 more
FAILED: org.apache.solr.cloud.HttpPartitionTest.test
Error Message:
Didn't see all replicas for shard shard1 in c8n_1x2 come up within 90000 ms! ClusterState: { "collMinRf_1x3":{ "pullReplicas":"0", "replicationFactor":"1", "shards":{"shard1":{ "range":"80000000-7fffffff", "state":"active", "replicas":{ "core_node1":{ "core":"collMinRf_1x3_shard1_replica_n1", "base_url":"http://127.0.0.1:36537", "node_name":"127.0.0.1:36537_", "state":"active", "type":"NRT"}, "core_node2":{ "core":"collMinRf_1x3_shard1_replica_n2", "base_url":"http://127.0.0.1:35393", "node_name":"127.0.0.1:35393_", "state":"active", "type":"NRT"}, "core_node3":{ "core":"collMinRf_1x3_shard1_replica_n3", "base_url":"http://127.0.0.1:36929", "node_name":"127.0.0.1:36929_", "state":"active", "type":"NRT", "leader":"true"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"3", "tlogReplicas":"0"}, "collection1":{ "pullReplicas":"0", "replicationFactor":"1", "shards":{ "shard1":{ "range":"80000000-ffffffff", "state":"active", "replicas":{"core_node2":{ "core":"collection1_shard1_replica_n0", "base_url":"http://127.0.0.1:41363", "node_name":"127.0.0.1:41363_", "state":"active", "type":"NRT", "leader":"true"}}}, "shard2":{ "range":"0-7fffffff", "state":"active", "replicas":{ "core_node1":{ "core":"collection1_shard2_replica_n0", "base_url":"http://127.0.0.1:36929", "node_name":"127.0.0.1:36929_", "state":"active", "type":"NRT", "leader":"true"}, "core_node3":{ "core":"collection1_shard2_replica_n1", "base_url":"http://127.0.0.1:35393", "node_name":"127.0.0.1:35393_", "state":"active", "type":"NRT"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"1", "tlogReplicas":"0"}, "control_collection":{ "pullReplicas":"0", "replicationFactor":"1", "shards":{"shard1":{ "range":"80000000-7fffffff", "state":"active", "replicas":{"core_node1":{ "core":"control_collection_shard1_replica_n1", "base_url":"http://127.0.0.1:36537", "node_name":"127.0.0.1:36537_", "state":"active", "type":"NRT", "leader":"true"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"1", "tlogReplicas":"0"}, "c8n_1x2":{ "pullReplicas":"0", "replicationFactor":"1", "shards":{"shard1":{ "range":"80000000-7fffffff", "state":"active", "replicas":{ "core_node1":{ "core":"c8n_1x2_shard1_replica_n1", "base_url":"http://127.0.0.1:36929", "node_name":"127.0.0.1:36929_", "state":"active", "type":"NRT", "leader":"true"}, "core_node2":{ "core":"c8n_1x2_shard1_replica_n2", "base_url":"http://127.0.0.1:41363", "node_name":"127.0.0.1:41363_", "state":"recovering", "type":"NRT"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"2", "tlogReplicas":"0"}}
Stack Trace:
java.lang.AssertionError: Didn't see all replicas for shard shard1 in c8n_1x2 come up within 90000 ms! ClusterState: {
"collMinRf_1x3":{
"pullReplicas":"0",
"replicationFactor":"1",
"shards":{"shard1":{
"range":"80000000-7fffffff",
"state":"active",
"replicas":{
"core_node1":{
"core":"collMinRf_1x3_shard1_replica_n1",
"base_url":"http://127.0.0.1:36537",
"node_name":"127.0.0.1:36537_",
"state":"active",
"type":"NRT"},
"core_node2":{
"core":"collMinRf_1x3_shard1_replica_n2",
"base_url":"http://127.0.0.1:35393",
"node_name":"127.0.0.1:35393_",
"state":"active",
"type":"NRT"},
"core_node3":{
"core":"collMinRf_1x3_shard1_replica_n3",
"base_url":"http://127.0.0.1:36929",
"node_name":"127.0.0.1:36929_",
"state":"active",
"type":"NRT",
"leader":"true"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"3",
"tlogReplicas":"0"},
"collection1":{
"pullReplicas":"0",
"replicationFactor":"1",
"shards":{
"shard1":{
"range":"80000000-ffffffff",
"state":"active",
"replicas":{"core_node2":{
"core":"collection1_shard1_replica_n0",
"base_url":"http://127.0.0.1:41363",
"node_name":"127.0.0.1:41363_",
"state":"active",
"type":"NRT",
"leader":"true"}}},
"shard2":{
"range":"0-7fffffff",
"state":"active",
"replicas":{
"core_node1":{
"core":"collection1_shard2_replica_n0",
"base_url":"http://127.0.0.1:36929",
"node_name":"127.0.0.1:36929_",
"state":"active",
"type":"NRT",
"leader":"true"},
"core_node3":{
"core":"collection1_shard2_replica_n1",
"base_url":"http://127.0.0.1:35393",
"node_name":"127.0.0.1:35393_",
"state":"active",
"type":"NRT"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"1",
"tlogReplicas":"0"},
"control_collection":{
"pullReplicas":"0",
"replicationFactor":"1",
"shards":{"shard1":{
"range":"80000000-7fffffff",
"state":"active",
"replicas":{"core_node1":{
"core":"control_collection_shard1_replica_n1",
"base_url":"http://127.0.0.1:36537",
"node_name":"127.0.0.1:36537_",
"state":"active",
"type":"NRT",
"leader":"true"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"1",
"tlogReplicas":"0"},
"c8n_1x2":{
"pullReplicas":"0",
"replicationFactor":"1",
"shards":{"shard1":{
"range":"80000000-7fffffff",
"state":"active",
"replicas":{
"core_node1":{
"core":"c8n_1x2_shard1_replica_n1",
"base_url":"http://127.0.0.1:36929",
"node_name":"127.0.0.1:36929_",
"state":"active",
"type":"NRT",
"leader":"true"},
"core_node2":{
"core":"c8n_1x2_shard1_replica_n2",
"base_url":"http://127.0.0.1:41363",
"node_name":"127.0.0.1:41363_",
"state":"recovering",
"type":"NRT"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"2",
"tlogReplicas":"0"}}
at __randomizedtesting.SeedInfo.seed([57BDCF540E9917CD:DFE9F08EA0657A35]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.ensureAllReplicasAreActive(AbstractFullDistribZkTestBase.java:1992)
at org.apache.solr.cloud.HttpPartitionTest.testRf2(HttpPartitionTest.java:367)
at org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:131)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest.test
Error Message:
The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
Stack Trace:
java.lang.AssertionError: The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
at __randomizedtesting.SeedInfo.seed([57BDCF540E9917CD:DFE9F08EA0657A35]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.ChaosMonkey.stopTheMonkey(ChaosMonkey.java:587)
at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test(ChaosMonkeySafeLeaderTest.java:133)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12100 lines...]
[junit4] Suite: org.apache.solr.cloud.HttpPartitionTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_57BDCF540E9917CD-001/init-core-data-001
[junit4] 2> 880765 WARN (SUITE-HttpPartitionTest-seed#[57BDCF540E9917CD]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
[junit4] 2> 880765 INFO (SUITE-HttpPartitionTest-seed#[57BDCF540E9917CD]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 880766 INFO (SUITE-HttpPartitionTest-seed#[57BDCF540E9917CD]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 880766 INFO (SUITE-HttpPartitionTest-seed#[57BDCF540E9917CD]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 880767 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 880768 INFO (Thread-2272) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 880768 INFO (Thread-2272) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 880769 ERROR (Thread-2272) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 880868 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.ZkTestServer start zk server on port:42343
[junit4] 2> 880877 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 880878 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 880880 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 880880 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 880881 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 880882 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 880883 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 880884 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 880885 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 880886 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 880887 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 880888 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 880962 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 880963 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6c156a75{/,null,AVAILABLE}
[junit4] 2> 880964 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@df0e738{HTTP/1.1,[http/1.1]}{127.0.0.1:40816}
[junit4] 2> 880964 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.e.j.s.Server Started @883040ms
[junit4] 2> 880964 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_57BDCF540E9917CD-001/tempDir-001/control/data, replicaType=NRT, hostContext=/, hostPort=36537, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_57BDCF540E9917CD-001/control-001/cores}
[junit4] 2> 880964 ERROR (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 880964 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.1.0
[junit4] 2> 880964 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 880964 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 880964 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-07-13T03:09:47.070Z
[junit4] 2> 880966 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 880966 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_57BDCF540E9917CD-001/control-001/solr.xml
[junit4] 2> 880971 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@28b35f7f, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 880973 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42343/solr
[junit4] 2> 881030 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [n:127.0.0.1:36537_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 881031 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [n:127.0.0.1:36537_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:36537_
[junit4] 2> 881031 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [n:127.0.0.1:36537_ ] o.a.s.c.Overseer Overseer (id=98298454794174468-127.0.0.1:36537_-n_0000000000) starting
[junit4] 2> 881036 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [n:127.0.0.1:36537_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36537_
[junit4] 2> 881037 INFO (zkCallback-1903-thread-1-processing-n:127.0.0.1:36537_) [n:127.0.0.1:36537_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 881092 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [n:127.0.0.1:36537_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@28b35f7f
[junit4] 2> 881096 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [n:127.0.0.1:36537_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@28b35f7f
[junit4] 2> 881096 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [n:127.0.0.1:36537_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@28b35f7f
[junit4] 2> 881097 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [n:127.0.0.1:36537_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_57BDCF540E9917CD-001/control-001/cores
[junit4] 2> 881111 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 881112 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42343/solr ready
[junit4] 2> 881113 INFO (SocketProxy-Acceptor-36537) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=52942,localport=36537], receiveBufferSize:531000
[junit4] 2> 881113 INFO (SocketProxy-Acceptor-36537) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=40816,localport=34446], receiveBufferSize=530904
[junit4] 2> 881115 INFO (qtp1976280674-9003) [n:127.0.0.1:36537_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:36537_&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 881117 INFO (OverseerThreadFactory-3120-thread-1-processing-n:127.0.0.1:36537_) [n:127.0.0.1:36537_ ] o.a.s.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 881220 INFO (SocketProxy-Acceptor-36537) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=52946,localport=36537], receiveBufferSize:531000
[junit4] 2> 881220 INFO (SocketProxy-Acceptor-36537) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=40816,localport=34450], receiveBufferSize=530904
[junit4] 2> 881223 INFO (qtp1976280674-9001) [n:127.0.0.1:36537_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 881223 INFO (qtp1976280674-9001) [n:127.0.0.1:36537_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 881326 INFO (zkCallback-1903-thread-1-processing-n:127.0.0.1:36537_) [n:127.0.0.1:36537_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 882233 INFO (qtp1976280674-9001) [n:127.0.0.1:36537_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 882244 INFO (qtp1976280674-9001) [n:127.0.0.1:36537_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 882324 INFO (qtp1976280674-9001) [n:127.0.0.1:36537_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 882335 INFO (qtp1976280674-9001) [n:127.0.0.1:36537_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 882335 INFO (qtp1976280674-9001) [n:127.0.0.1:36537_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@28b35f7f
[junit4] 2> 882335 INFO (qtp1976280674-9001) [n:127.0.0.1:36537_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 882335 INFO (qtp1976280674-9001) [n:127.0.0.1:36537_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_57BDCF540E9917CD-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_57BDCF540E9917CD-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 882337 INFO (qtp1976280674-9001) [n:127.0.0.1:36537_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=7, maxMergeAtOnceExplicit=5, maxMergedSegmentMB=8.29296875, floorSegmentMB=1.4375, forceMergeDeletesPctAllowed=7.88214974082647, segmentsPerTier=30.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.477010148973108
[junit4] 2> 882339 WARN (qtp1976280674-9001) [n:127.0.0.1:36537_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 882370 INFO (qtp1976280674-9001) [n:127.0.0.1:36537_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 882370 INFO (qtp1976280674-9001) [n:127.0.0.1:36537_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 882371 INFO (qtp1976280674-9001) [n:127.0.0.1:36537_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 882372 INFO (qtp1976280674-9001) [n:127.0.0.1:36537_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 882372 INFO (qtp1976280674-9001) [n:127.0.0.1:36537_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1005444240, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 882373 INFO (qtp1976280674-9001) [n:127.0.0.1:36537_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2ace77c7[control_collection_shard1_replica_n1] main]
[junit4] 2> 882374 INFO (qtp1976280674-9001) [n:127.0.0.1:36537_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 882374 INFO (qtp1976280674-9001) [n:127.0.0.1:36537_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 882374 INFO (qtp1976280674-9001) [n:127.0.0.1:36537_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 882375 INFO (searcherExecutor-3123-thread-1-processing-n:127.0.0.1:36537_ x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:36537_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@2ace77c7[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 882375 INFO (qtp1976280674-9001) [n:127.0.0.1:36537_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1572775278391853056
[junit4] 2> 882380 INFO (qtp1976280674-9001) [n:127.0.0.1:36537_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 882380 INFO (qtp1976280674-9001) [n:127.0.0.1:36537_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 882380 INFO (qtp1976280674-9001) [n:127.0.0.1:36537_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:36537/control_collection_shard1_replica_n1/
[junit4] 2> 882380 INFO (qtp1976280674-9001) [n:127.0.0.1:36537_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 882380 INFO (qtp1976280674-9001) [n:127.0.0.1:36537_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:36537/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 882380 INFO (qtp1976280674-9001) [n:127.0.0.1:36537_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 882383 INFO (qtp1976280674-9001) [n:127.0.0.1:36537_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:36537/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 882484 INFO (zkCallback-1903-thread-1-processing-n:127.0.0.1:36537_) [n:127.0.0.1:36537_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 882533 INFO (qtp1976280674-9001) [n:127.0.0.1:36537_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 882535 INFO (qtp1976280674-9001) [n:127.0.0.1:36537_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1312
[junit4] 2> 882544 INFO (qtp1976280674-9003) [n:127.0.0.1:36537_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 882643 INFO (zkCallback-1903-thread-1-processing-n:127.0.0.1:36537_) [n:127.0.0.1:36537_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 883119 INFO (OverseerCollectionConfigSetProcessor-98298454794174468-127.0.0.1:36537_-n_0000000000) [n:127.0.0.1:36537_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 883545 INFO (qtp1976280674-9003) [n:127.0.0.1:36537_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:36537_&wt=javabin&version=2} status=0 QTime=2430
[junit4] 2> 883550 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 883550 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42343/solr ready
[junit4] 2> 883550 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 883551 INFO (SocketProxy-Acceptor-36537) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=52964,localport=36537], receiveBufferSize:531000
[junit4] 2> 883551 INFO (SocketProxy-Acceptor-36537) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=40816,localport=34468], receiveBufferSize=530904
[junit4] 2> 883552 INFO (qtp1976280674-9007) [n:127.0.0.1:36537_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 883554 INFO (OverseerThreadFactory-3120-thread-2-processing-n:127.0.0.1:36537_) [n:127.0.0.1:36537_ ] o.a.s.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 883555 WARN (OverseerThreadFactory-3120-thread-2-processing-n:127.0.0.1:36537_) [n:127.0.0.1:36537_ ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 883758 INFO (qtp1976280674-9007) [n:127.0.0.1:36537_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 883758 INFO (qtp1976280674-9007) [n:127.0.0.1:36537_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2} status=0 QTime=206
[junit4] 2> 883832 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_57BDCF540E9917CD-001/shard-1-001 of type NRT
[junit4] 2> 883833 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 883834 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4fbfce95{/,null,AVAILABLE}
[junit4] 2> 883834 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3b542add{HTTP/1.1,[http/1.1]}{127.0.0.1:37851}
[junit4] 2> 883834 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.e.j.s.Server Started @885911ms
[junit4] 2> 883834 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_57BDCF540E9917CD-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=36929, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_57BDCF540E9917CD-001/shard-1-001/cores}
[junit4] 2> 883834 ERROR (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 883834 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.1.0
[junit4] 2> 883834 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 883834 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 883834 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-07-13T03:09:49.940Z
[junit4] 2> 883836 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 883836 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_57BDCF540E9917CD-001/shard-1-001/solr.xml
[junit4] 2> 883840 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@28b35f7f, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 883842 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42343/solr
[junit4] 2> 883848 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [n:127.0.0.1:36929_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 883850 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [n:127.0.0.1:36929_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 883851 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [n:127.0.0.1:36929_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36929_
[junit4] 2> 883852 INFO (zkCallback-1910-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 883852 INFO (zkCallback-1903-thread-1-processing-n:127.0.0.1:36537_) [n:127.0.0.1:36537_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 883852 INFO (zkCallback-1915-thread-1-processing-n:127.0.0.1:36929_) [n:127.0.0.1:36929_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 883904 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [n:127.0.0.1:36929_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@28b35f7f
[junit4] 2> 883908 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [n:127.0.0.1:36929_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@28b35f7f
[junit4] 2> 883908 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [n:127.0.0.1:36929_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@28b35f7f
[junit4] 2> 883909 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [n:127.0.0.1:36929_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_57BDCF540E9917CD-001/shard-1-001/cores
[junit4] 2> 883926 INFO (SocketProxy-Acceptor-36929) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=46180,localport=36929], receiveBufferSize:531000
[junit4] 2> 883926 INFO (SocketProxy-Acceptor-36929) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=37851,localport=39518], receiveBufferSize=530904
[junit4] 2> 883927 INFO (qtp421365138-9051) [n:127.0.0.1:36929_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:36929_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 883929 INFO (OverseerCollectionConfigSetProcessor-98298454794174468-127.0.0.1:36537_-n_0000000000) [n:127.0.0.1:36537_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 883929 INFO (OverseerThreadFactory-3120-thread-3-processing-n:127.0.0.1:36537_) [n:127.0.0.1:36537_ ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:36929_ for creating new replica
[junit4] 2> 883930 INFO (SocketProxy-Acceptor-36929) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=46184,localport=36929], receiveBufferSize:531000
[junit4] 2> 883930 INFO (SocketProxy-Acceptor-36929) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=37851,localport=39522], receiveBufferSize=530904
[junit4] 2> 883930 INFO (qtp421365138-9053) [n:127.0.0.1:36929_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n0&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 883931 INFO (qtp421365138-9053) [n:127.0.0.1:36929_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 884033 INFO (zkCallback-1915-thread-1-processing-n:127.0.0.1:36929_) [n:127.0.0.1:36929_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 884941 INFO (qtp421365138-9053) [n:127.0.0.1:36929_ c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 884951 INFO (qtp421365138-9053) [n:127.0.0.1:36929_ c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.s.IndexSchema [collection1_shard2_replica_n0] Schema name=test
[junit4] 2> 885034 INFO (qtp421365138-9053) [n:127.0.0.1:36929_ c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 885046 INFO (qtp421365138-9053) [n:127.0.0.1:36929_ c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n0' using configuration from collection collection1, trusted=true
[junit4] 2> 885046 INFO (qtp421365138-9053) [n:127.0.0.1:36929_ c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n0' (registry 'solr.core.collection1.shard2.replica_n0') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@28b35f7f
[junit4] 2> 885046 INFO (qtp421365138-9053) [n:127.0.0.1:36929_ c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 885046 INFO (qtp421365138-9053) [n:127.0.0.1:36929_ c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.SolrCore [[collection1_shard2_replica_n0] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_57BDCF540E9917CD-001/shard-1-001/cores/collection1_shard2_replica_n0], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_57BDCF540E9917CD-001/shard-1-001/cores/collection1_shard2_replica_n0/data/]
[junit4] 2> 885048 INFO (qtp421365138-9053) [n:127.0.0.1:36929_ c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=7, maxMergeAtOnceExplicit=5, maxMergedSegmentMB=8.29296875, floorSegmentMB=1.4375, forceMergeDeletesPctAllowed=7.88214974082647, segmentsPerTier=30.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.477010148973108
[junit4] 2> 885050 WARN (qtp421365138-9053) [n:127.0.0.1:36929_ c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 885082 INFO (qtp421365138-9053) [n:127.0.0.1:36929_ c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 885082 INFO (qtp421365138-9053) [n:127.0.0.1:36929_ c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 885083 INFO (qtp421365138-9053) [n:127.0.0.1:36929_ c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 885083 INFO (qtp421365138-9053) [n:127.0.0.1:36929_ c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 885084 INFO (qtp421365138-9053) [n:127.0.0.1:36929_ c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1005444240, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 885084 INFO (qtp421365138-9053) [n:127.0.0.1:36929_ c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.s.SolrIndexSearcher Opening [Searcher@721f0441[collection1_shard2_replica_n0] main]
[junit4] 2> 885085 INFO (qtp421365138-9053) [n:127.0.0.1:36929_ c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 885086 INFO (qtp421365138-9053) [n:127.0.0.1:36929_ c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 885086 INFO (qtp421365138-9053) [n:127.0.0.1:36929_ c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 885087 INFO (searcherExecutor-3134-thread-1-processing-n:127.0.0.1:36929_ x:collection1_shard2_replica_n0 s:shard2 c:collection1) [n:127.0.0.1:36929_ c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.SolrCore [collection1_shard2_replica_n0] Registered new searcher Searcher@721f0441[collection1_shard2_replica_n0] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 885087 INFO (qtp421365138-9053) [n:127.0.0.1:36929_ c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1572775281235591168
[junit4] 2> 885095 INFO (qtp421365138-9053) [n:127.0.0.1:36929_ c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 885095 INFO (qtp421365138-9053) [n:127.0.0.1:36929_ c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 885095 INFO (qtp421365138-9053) [n:127.0.0.1:36929_ c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:36929/collection1_shard2_replica_n0/
[junit4] 2> 885095 INFO (qtp421365138-9053) [n:127.0.0.1:36929_ c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 885095 INFO (qtp421365138-9053) [n:127.0.0.1:36929_ c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.SyncStrategy http://127.0.0.1:36929/collection1_shard2_replica_n0/ has no replicas
[junit4] 2> 885095 INFO (qtp421365138-9053) [n:127.0.0.1:36929_ c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 885097 INFO (qtp421365138-9053) [n:127.0.0.1:36929_ c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:36929/collection1_shard2_replica_n0/ shard2
[junit4] 2> 885199 INFO (zkCallback-1915-thread-1-processing-n:127.0.0.1:36929_) [n:127.0.0.1:36929_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 885248 INFO (qtp421365138-9053) [n:127.0.0.1:36929_ c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 885250 INFO (qtp421365138-9053) [n:127.0.0.1:36929_ c:collection1 s:shard2 x:collection1_shard2_replica_n0] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n0&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1319
[junit4] 2> 885251 INFO (qtp421365138-9051) [n:127.0.0.1:36929_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:36929_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2} status=0 QTime=1324
[junit4] 2> 885327 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_57BDCF540E9917CD-001/shard-2-001 of type NRT
[junit4] 2> 885327 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 885328 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5bb73dfd{/,null,AVAILABLE}
[junit4] 2> 885328 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7a01e966{HTTP/1.1,[http/1.1]}{127.0.0.1:32870}
[junit4] 2> 885328 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.e.j.s.Server Started @887405ms
[junit4] 2> 885328 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_57BDCF540E9917CD-001/tempDir-001/jetty2, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/, hostPort=41363, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_57BDCF540E9917CD-001/shard-2-001/cores}
[junit4] 2> 885328 ERROR (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 885329 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.1.0
[junit4] 2> 885329 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 885329 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 885329 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-07-13T03:09:51.435Z
[junit4] 2> 885331 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 885331 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_57BDCF540E9917CD-001/shard-2-001/solr.xml
[junit4] 2> 885335 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@28b35f7f, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 885337 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42343/solr
[junit4] 2> 885343 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [n:127.0.0.1:41363_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 885344 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [n:127.0.0.1:41363_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 885345 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [n:127.0.0.1:41363_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41363_
[junit4] 2> 885346 INFO (zkCallback-1903-thread-1-processing-n:127.0.0.1:36537_) [n:127.0.0.1:36537_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 885346 INFO (zkCallback-1915-thread-1-processing-n:127.0.0.1:36929_) [n:127.0.0.1:36929_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 885346 INFO (zkCallback-1910-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 885347 INFO (zkCallback-1921-thread-1-processing-n:127.0.0.1:41363_) [n:127.0.0.1:41363_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 885448 INFO (zkCallback-1915-thread-1-processing-n:127.0.0.1:36929_) [n:127.0.0.1:36929_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 885502 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [n:127.0.0.1:41363_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@28b35f7f
[junit4] 2> 885506 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [n:127.0.0.1:41363_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@28b35f7f
[junit4] 2> 885506 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [n:127.0.0.1:41363_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@28b35f7f
[junit4] 2> 885507 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [n:127.0.0.1:41363_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_57BDCF540E9917CD-001/shard-2-001/cores
[junit4] 2> 885536 INFO (SocketProxy-Acceptor-41363) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=37364,localport=41363], receiveBufferSize:531000
[junit4] 2> 885537 INFO (SocketProxy-Acceptor-41363) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=32870,localport=47114], receiveBufferSize=530904
[junit4] 2> 885538 INFO (qtp87552865-9085) [n:127.0.0.1:41363_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:41363_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 885540 INFO (OverseerCollectionConfigSetProcessor-98298454794174468-127.0.0.1:36537_-n_0000000000) [n:127.0.0.1:36537_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 885540 INFO (OverseerThreadFactory-3120-thread-4-processing-n:127.0.0.1:36537_) [n:127.0.0.1:36537_ ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:41363_ for creating new replica
[junit4] 2> 885541 INFO (SocketProxy-Acceptor-41363) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=37368,localport=41363], receiveBufferSize:531000
[junit4] 2> 885541 INFO (SocketProxy-Acceptor-41363) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=32870,localport=47118], receiveBufferSize=530904
[junit4] 2> 885542 INFO (qtp87552865-9087) [n:127.0.0.1:41363_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n0&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 885542 INFO (qtp87552865-9087) [n:127.0.0.1:41363_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 885645 INFO (zkCallback-1915-thread-1-processing-n:127.0.0.1:36929_) [n:127.0.0.1:36929_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 885645 INFO (zkCallback-1921-thread-1-processing-n:127.0.0.1:41363_) [n:127.0.0.1:41363_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 886553 INFO (qtp87552865-9087) [n:127.0.0.1:41363_ c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 886563 INFO (qtp87552865-9087) [n:127.0.0.1:41363_ c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.s.IndexSchema [collection1_shard1_replica_n0] Schema name=test
[junit4] 2> 886639 INFO (qtp87552865-9087) [n:127.0.0.1:41363_ c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 886650 INFO (qtp87552865-9087) [n:127.0.0.1:41363_ c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n0' using configuration from collection collection1, trusted=true
[junit4] 2> 886650 INFO (qtp87552865-9087) [n:127.0.0.1:41363_ c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n0' (registry 'solr.core.collection1.shard1.replica_n0') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@28b35f7f
[junit4] 2> 886650 INFO (qtp87552865-9087) [n:127.0.0.1:41363_ c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 886651 INFO (qtp87552865-9087) [n:127.0.0.1:41363_ c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.SolrCore [[collection1_shard1_replica_n0] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_57BDCF540E9917CD-001/shard-2-001/cores/collection1_shard1_replica_n0], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_57BDCF540E9917CD-001/shard-2-001/cores/collection1_shard1_replica_n0/data/]
[junit4] 2> 886652 INFO (qtp87552865-9087) [n:127.0.0.1:41363_ c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=7, maxMergeAtOnceExplicit=5, maxMergedSegmentMB=8.29296875, floorSegmentMB=1.4375, forceMergeDeletesPctAllowed=7.88214974082647, segmentsPerTier=30.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.477010148973108
[junit4] 2> 886654 WARN (qtp87552865-9087) [n:127.0.0.1:41363_ c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 886685 INFO (qtp87552865-9087) [n:127.0.0.1:41363_ c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 886685 INFO (qtp87552865-9087) [n:127.0.0.1:41363_ c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 886686 INFO (qtp87552865-9087) [n:127.0.0.1:41363_ c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 886686 INFO (qtp87552865-9087) [n:127.0.0.1:41363_ c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 886687 INFO (qtp87552865-9087) [n:127.0.0.1:41363_ c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1005444240, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 886687 INFO (qtp87552865-9087) [n:127.0.0.1:41363_ c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.s.SolrIndexSearcher Opening [Searcher@131f1b04[collection1_shard1_replica_n0] main]
[junit4] 2> 886688 INFO (qtp87552865-9087) [n:127.0.0.1:41363_ c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 886688 INFO (qtp87552865-9087) [n:127.0.0.1:41363_ c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 886689 INFO (qtp87552865-9087) [n:127.0.0.1:41363_ c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 886690 INFO (searcherExecutor-3145-thread-1-processing-n:127.0.0.1:41363_ x:collection1_shard1_replica_n0 s:shard1 c:collection1) [n:127.0.0.1:41363_ c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.SolrCore [collection1_shard1_replica_n0] Registered new searcher Searcher@131f1b04[collection1_shard1_replica_n0] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 886690 INFO (qtp87552865-9087) [n:127.0.0.1:41363_ c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1572775282916458496
[junit4] 2> 886694 INFO (qtp87552865-9087) [n:127.0.0.1:41363_ c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 886694 INFO (qtp87552865-9087) [n:127.0.0.1:41363_ c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 886694 INFO (qtp87552865-9087) [n:127.0.0.1:41363_ c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:41363/collection1_shard1_replica_n0/
[junit4] 2> 886694 INFO (qtp87552865-9087) [n:127.0.0.1:41363_ c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 886694 INFO (qtp87552865-9087) [n:127.0.0.1:41363_ c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.SyncStrategy http://127.0.0.1:41363/collection1_shard1_replica_n0/ has no replicas
[junit4] 2> 886694 INFO (qtp87552865-9087) [n:127.0.0.1:41363_ c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 886697 INFO (qtp87552865-9087) [n:127.0.0.1:41363_ c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:41363/collection1_shard1_replica_n0/ shard1
[junit4] 2> 886799 INFO (zkCallback-1921-thread-1-processing-n:127.0.0.1:41363_) [n:127.0.0.1:41363_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 886799 INFO (zkCallback-1915-thread-1-processing-n:127.0.0.1:36929_) [n:127.0.0.1:36929_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 886848 INFO (qtp87552865-9087) [n:127.0.0.1:41363_ c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 886849 INFO (qtp87552865-9087) [n:127.0.0.1:41363_ c:collection1 s:shard1 x:collection1_shard1_replica_n0] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n0&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1307
[junit4] 2> 886851 INFO (qtp87552865-9085) [n:127.0.0.1:41363_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:41363_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=1312
[junit4] 2> 886927 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_57BDCF540E9917CD-001/shard-3-001 of type NRT
[junit4] 2> 886927 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 886928 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7449ad3c{/,null,AVAILABLE}
[junit4] 2> 886928 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@36a4c247{HTTP/1.1,[http/1.1]}{127.0.0.1:36214}
[junit4] 2> 886928 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.e.j.s.Server Started @889005ms
[junit4] 2> 886928 INFO (TEST-HttpPartitionTest.test-seed#[57BDCF540E9917CD]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_57BDCF540E9917CD-001/tempDir-001/jetty3, solrconfig=solrconfig.xml, hostContext=/, hostPort=35393, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_57BDCF540E9917CD-001/shard-3-001/cores}
[junit4] 2> 886929 ERROR (TEST-HttpPartitionTest.test-seed#[57BDCF54
[...truncated too long message...]
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:872)
[junit4] 2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1726)
[junit4] 2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1705)
[junit4] 2> at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:204)
[junit4] 2> at org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest.teardownClass(HdfsChaosMonkeySafeLeaderTest.java:47)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: java.lang.NullPointerException
[junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:203)
[junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3370)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:5729)
[junit4] 2> ... 54 more
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_57BDCF540E9917CD-001
[junit4] 2> Jul 13, 2017 3:44:56 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 34 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {rnd_b=PostingsFormat(name=Asserting), a_t=PostingsFormat(name=LuceneVarGapFixedInterval), a_i=PostingsFormat(name=Asserting), id=PostingsFormat(name=Asserting)}, docValues:{_version_=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=463, maxMBSortInHeap=6.128585903373934, sim=RandomSimilarity(queryNorm=true): {}, locale=es-CO, timezone=Africa/Bissau
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_131 (64-bit)/cpus=4,threads=2,free=165640360,total=529006592
[junit4] 2> NOTE: All tests run in this JVM: [SmileWriterTest, TestMultiWordSynonyms, ZkNodePropsTest, UUIDFieldTest, TestPayloadCheckQParserPlugin, TestTolerantUpdateProcessorRandomCloud, TestWriterPerf, TestNestedDocsSort, TestCustomStream, TestJavabinTupleStreamParser, TestFieldCacheVsDocValues, DeleteShardTest, TestRangeQuery, TestHighlightDedupGrouping, TestSSLRandomization, OverseerStatusTest, TestStressUserVersions, OutputWriterTest, RecoveryAfterSoftCommitTest, TestConfigsApi, SolrCLIZkUtilsTest, CoreMergeIndexesAdminHandlerTest, TestSchemaNameResource, ClassificationUpdateProcessorTest, ResponseHeaderTest, SpellCheckCollatorTest, TestSolrCLIRunExample, TestConfigSetProperties, TestSolrQueryParser, PluginInfoTest, TestFaceting, TestInPlaceUpdatesDistrib, PropertiesRequestHandlerTest, FieldAnalysisRequestHandlerTest, TestSolrCoreSnapshots, TestSQLHandler, TolerantUpdateProcessorTest, DefaultValueUpdateProcessorTest, DistributedVersionInfoTest, TestNumericTokenStream, TestBlobHandler, TestLocalFSCloudBackupRestore, CurrencyFieldTypeTest, HdfsLockFactoryTest, AtomicUpdatesTest, BlockJoinFacetSimpleTest, SSLMigrationTest, TestSizeLimitedDistributedMap, TestBackupRepositoryFactory, TestDownShardTolerantSearch, HLLSerializationTest, VMParamsZkACLAndCredentialsProvidersTest, EnumFieldTest, DistributedSpellCheckComponentTest, TestSolrConfigHandlerConcurrent, TestManagedResource, TestReplicationHandler, BlobRepositoryCloudTest, TestImpersonationWithHadoopAuth, DistributedFacetPivotWhiteBoxTest, RequestHandlersTest, TestReloadAndDeleteDocs, TestXmlQParserPlugin, TestLRUStatsCache, OpenExchangeRatesOrgProviderTest, TestSchemaManager, NodeMutatorTest, TestLegacyNumericRangeQueryBuilder, TestRawTransformer, MetricUtilsTest, TestRealTimeGet, TestCollapseQParserPlugin, CdcrRequestHandlerTest, TestIndexingPerformance, HighlighterConfigTest, HttpPartitionTest, TestComponentsName, TestCloudManagedSchema, BadCopyFieldTest, TestRuleBasedAuthorizationPlugin, DebugComponentTest, TestDistributedSearch, TestLeaderElectionWithEmptyReplica, HdfsSyncSliceTest, SpellingQueryConverterTest, ReplicationFactorTest, TestOmitPositions, TestSolrCoreParser, TestSolr4Spatial, RankQueryTest, TestSimpleQParserPlugin, LeaderFailureAfterFreshStartTest, ConnectionReuseTest, TestReplicationHandlerBackup, SolrPluginUtilsTest, TestSolrConfigHandler, SolrIndexConfigTest, TestJsonRequest, CursorMarkTest, StatsReloadRaceTest, TestLegacyFieldReuse, TestMergePolicyConfig, TestFreeTextSuggestions, TestSolrDeletionPolicy1, ExitableDirectoryReaderTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, RAMDirectoryFactoryTest, TestUseDocValuesAsStored2, TestSchemaVersionResource, WrapperMergePolicyFactoryTest, FieldMutatingUpdateProcessorTest, TestConfig, V2StandaloneTest, MBeansHandlerTest, ImplicitSnitchTest, MetricsConfigTest, MigrateRouteKeyTest, LeaderElectionIntegrationTest, BasicZkTest, FullSolrCloudDistribCmdsTest, ClusterStateUpdateTest, TestRandomFaceting, ZkSolrClientTest, TestRandomDVFaceting, TestReload, SolrCmdDistributorTest, QueryElevationComponentTest, PeerSyncTest, TestLazyCores, DistributedQueryElevationComponentTest, SolrIndexSplitterTest, SimplePostToolTest, TestCoreDiscovery, AnalysisAfterCoreReloadTest, TestExtendedDismaxParser, SolrRequestParserTest, TestFoldingMultitermQuery, SuggesterTest, TestStressLucene, SpatialFilterTest, SuggesterWFSTTest, TestCSVLoader, PolyFieldTest, WordBreakSolrSpellCheckerTest, SolrCoreCheckLockOnStartupTest, TestPseudoReturnFields, TestUpdate, TestAtomicUpdateErrorCases, TestWordDelimiterFilterFactory, SortByFunctionTest, TestRemoteStreaming, DocValuesMultiTest, SolrInfoBeanTest, IndexBasedSpellCheckerTest, DisMaxRequestHandlerTest, TestQueryUtils, DirectSolrSpellCheckerTest, TestReversedWildcardFilterFactory, DocumentAnalysisRequestHandlerTest, TestQueryTypes, XmlUpdateRequestHandlerTest, TermVectorComponentTest, MoreLikeThisHandlerTest, RequiredFieldsTest, LoggingHandlerTest, RegexBoostProcessorTest, IndexSchemaTest, TestCollationField, ReturnFieldsTest, TestCSVResponseWriter, QueryParsingTest, TestAnalyzedSuggestions, JsonLoaderTest, BinaryUpdateRequestHandlerTest, TestPHPSerializedResponseWriter, PingRequestHandlerTest, SampleTest, TestDocSet, TestBinaryField, NumericFieldsTest, MinimalSchemaTest, TestFuzzyAnalyzedSuggestions, ExternalFileFieldSortTest, TestSolrCoreProperties, TestLuceneMatchVersion, TestPhraseSuggestions, SpellPossibilityIteratorTest, TestCharFilters, SynonymTokenizerTest, EchoParamsTest, TestSweetSpotSimilarityFactory, TestLMDirichletSimilarityFactory, TestLMJelinekMercerSimilarityFactory, TestIBSimilarityFactory, TimeZoneUtilsTest, TestFastOutputStream, URLClassifyProcessorTest, ChaosMonkeyNothingIsSafeTest, CdcrVersionReplicationTest, ChaosMonkeySafeLeaderWithPullReplicasTest, CleanupOldIndexTest, CloudExitableDirectoryReaderTest, CollectionStateFormat2Test, CollectionTooManyReplicasTest, CollectionsAPISolrJTest, ConfigSetsAPITest, ConnectionManagerTest, CreateCollectionCleanupTest, CustomCollectionTest, DeleteStatusTest, DistribJoinFromCollectionTest, ForceLeaderTest, MoveReplicaTest, RestartWhileUpdatingTest, TestCloudPseudoReturnFields, TestClusterProperties, TestConfigSetsAPI, TestOnReconnectListenerSupport, TestRandomFlRTGCloud, TestStressLiveNodes, AutoScalingHandlerTest, TestPolicyCloud, HdfsBasicDistributedZk2Test, HdfsChaosMonkeySafeLeaderTest]
[junit4] Completed [568/728 (4!)] on J0 in 83.83s, 1 test, 1 failure <<< FAILURES!
[...truncated 43820 lines...]
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 7 - Still Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/7/
6 tests failed.
FAILED: org.apache.lucene.index.TestIndexSorting.testRandom3
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([33668C7C0492929E]:0)
FAILED: junit.framework.TestSuite.org.apache.lucene.index.TestIndexSorting
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([33668C7C0492929E]:0)
FAILED: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test
Error Message:
Java heap space
Stack Trace:
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:3332)
at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124)
at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:649)
at java.lang.StringBuilder.append(StringBuilder.java:202)
at org.apache.http.client.utils.URLEncodedUtils.urlEncode(URLEncodedUtils.java:522)
at org.apache.http.client.utils.URLEncodedUtils.encodeFormFields(URLEncodedUtils.java:638)
at org.apache.http.client.utils.URLEncodedUtils.format(URLEncodedUtils.java:390)
at org.apache.http.client.utils.URLEncodedUtils.format(URLEncodedUtils.java:369)
at org.apache.http.client.entity.UrlEncodedFormEntity.<init>(UrlEncodedFormEntity.java:77)
at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:447)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:250)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1121)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:974)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:990)
at org.apache.solr.cloud.CloudInspectUtil.compareResults(CloudInspectUtil.java:228)
at org.apache.solr.cloud.CloudInspectUtil.compareResults(CloudInspectUtil.java:167)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testIndexingBatchPerRequestWithHttpSolrClient(FullSolrCloudDistribCmdsTest.java:668)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test(FullSolrCloudDistribCmdsTest.java:152)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
20 threads leaked from SUITE scope at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest: 1) Thread[id=20390, name=searcherExecutor-5760-thread-1, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=20080, name=qtp1812211277-20080, state=BLOCKED, group=TGRP-FullSolrCloudDistribCmdsTest] at org.apache.log4j.Category.callAppenders(Category.java:204) at org.apache.log4j.Category.forcedLog(Category.java:391) at org.apache.log4j.Category.log(Category.java:856) at org.slf4j.impl.Log4jLoggerAdapter.info(Log4jLoggerAdapter.java:304) at org.apache.solr.core.SolrCore.execute(SolrCore.java:2478) at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:720) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:526) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:378) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:322) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:534) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=20402, name=searcherExecutor-5763-thread-1, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=20389, name=searcherExecutor-5761-thread-1, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=20111, name=qtp150412892-20111, state=BLOCKED, group=TGRP-FullSolrCloudDistribCmdsTest] at org.apache.lucene.store.MockIndexInputWrapper.readByte(MockIndexInputWrapper.java:140) at org.apache.lucene.codecs.compressing.LZ4.decompress(LZ4.java:110) at org.apache.lucene.codecs.compressing.CompressionMode$4.decompress(CompressionMode.java:138) at org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader$BlockState.document(CompressingStoredFieldsReader.java:555) at org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.document(CompressingStoredFieldsReader.java:571) at org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.visitDocument(CompressingStoredFieldsReader.java:578) at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat$AssertingStoredFieldsReader.visitDocument(AssertingStoredFieldsFormat.java:73) at org.apache.lucene.index.CodecReader.document(CodecReader.java:83) at org.apache.lucene.index.FilterLeafReader.document(FilterLeafReader.java:341) at org.apache.lucene.index.FilterLeafReader.document(FilterLeafReader.java:341) at org.apache.lucene.index.BaseCompositeReader.document(BaseCompositeReader.java:118) at org.apache.solr.search.SolrDocumentFetcher.doc(SolrDocumentFetcher.java:198) at org.apache.solr.response.DocsStreamer.next(DocsStreamer.java:155) at org.apache.solr.response.DocsStreamer.next(DocsStreamer.java:59) at org.apache.solr.response.BinaryResponseWriter$Resolver.writeResultsBody(BinaryResponseWriter.java:126) at org.apache.solr.response.BinaryResponseWriter$Resolver.writeResults(BinaryResponseWriter.java:145) at org.apache.solr.response.BinaryResponseWriter$Resolver.resolve(BinaryResponseWriter.java:89) at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:239) at org.apache.solr.common.util.JavaBinCodec.writeNamedList(JavaBinCodec.java:223) at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:330) at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228) at org.apache.solr.common.util.JavaBinCodec.marshal(JavaBinCodec.java:155) at org.apache.solr.response.BinaryResponseWriter.write(BinaryResponseWriter.java:52) at org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:49) at org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:806) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:535) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:378) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:322) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:534) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=20398, name=searcherExecutor-5765-thread-1, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) 7) Thread[id=20525, name=qtp150412892-20525, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest] at java.io.FilterOutputStream.write(FilterOutputStream.java:125) at org.apache.lucene.util.TestRuleLimitSysouts$DelegateStream.write(TestRuleLimitSysouts.java:128) at java.io.PrintStream.write(PrintStream.java:480) at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221) at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:282) at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125) at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:135) at java.io.OutputStreamWriter.write(OutputStreamWriter.java:220) at java.io.Writer.write(Writer.java:157) at org.apache.log4j.helpers.QuietWriter.write(QuietWriter.java:48) at org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:310) at org.apache.log4j.WriterAppender.append(WriterAppender.java:162) at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251) at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66) at org.apache.log4j.Category.callAppenders(Category.java:206) at org.apache.log4j.Category.forcedLog(Category.java:391) at org.apache.log4j.Category.log(Category.java:856) at org.slf4j.impl.Log4jLoggerAdapter.info(Log4jLoggerAdapter.java:304) at org.apache.solr.core.SolrCore.execute(SolrCore.java:2478) at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:720) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:526) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:378) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:322) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:534) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 8) Thread[id=20514, name=qtp423659769-20514, state=BLOCKED, group=TGRP-FullSolrCloudDistribCmdsTest] at org.apache.log4j.Category.callAppenders(Category.java:204) at org.apache.log4j.Category.forcedLog(Category.java:391) at org.apache.log4j.Category.log(Category.java:856) at org.slf4j.impl.Log4jLoggerAdapter.info(Log4jLoggerAdapter.java:304) at org.apache.solr.core.SolrCore.execute(SolrCore.java:2478) at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:720) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:526) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:378) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:322) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:534) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 9) Thread[id=20489, name=qtp423659769-20489, state=BLOCKED, group=TGRP-FullSolrCloudDistribCmdsTest] at org.apache.log4j.Category.callAppenders(Category.java:204) at org.apache.log4j.Category.forcedLog(Category.java:391) at org.apache.log4j.Category.log(Category.java:856) at org.slf4j.impl.Log4jLoggerAdapter.info(Log4jLoggerAdapter.java:381) at org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:810) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:535) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:378) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:322) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:534) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 10) Thread[id=20501, name=qtp1812211277-20501, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest] at java.util.Arrays.copyOfRange(Arrays.java:3664) at java.lang.String.<init>(String.java:207) at java.lang.StringBuilder.toString(StringBuilder.java:407) at org.apache.lucene.store.MockIndexInputWrapper.<init>(MockIndexInputWrapper.java:40) at org.apache.lucene.store.MockIndexInputWrapper.clone(MockIndexInputWrapper.java:89) at org.apache.lucene.store.MockIndexInputWrapper.clone(MockIndexInputWrapper.java:29) at org.apache.lucene.codecs.blockterms.BlockTermsReader$FieldReader$SegmentTermsEnum.<init>(BlockTermsReader.java:331) at org.apache.lucene.codecs.blockterms.BlockTermsReader$FieldReader.iterator(BlockTermsReader.java:243) at org.apache.lucene.search.TermQuery$TermWeight.getTermsEnum(TermQuery.java:121) at org.apache.lucene.search.TermQuery$TermWeight.scorer(TermQuery.java:90) at org.apache.lucene.search.Weight.bulkScorer(Weight.java:147) at org.apache.lucene.search.BooleanWeight.optionalBulkScorer(BooleanWeight.java:157) at org.apache.lucene.search.BooleanWeight.booleanScorer(BooleanWeight.java:214) at org.apache.lucene.search.BooleanWeight.bulkScorer(BooleanWeight.java:283) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:655) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:462) at org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:215) at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1591) at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1408) at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:575) at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:544) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177) at org.apache.solr.core.SolrCore.execute(SolrCore.java:2473) at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:720) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:526) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:378) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:322) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:534) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 11) Thread[id=20400, name=searcherExecutor-5773-thread-1, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) 12) Thread[id=20401, name=searcherExecutor-5767-thread-1, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) 13) Thread[id=20393, name=searcherExecutor-5769-thread-1, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) 14) Thread[id=20410, name=searcherExecutor-5771-thread-1, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) 15) Thread[id=20166, name=qtp2106581206-20166, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest] at java.lang.String.valueOf(String.java:2994) at java.lang.StringBuilder.append(StringBuilder.java:131) at org.apache.lucene.store.MockIndexInputWrapper.<init>(MockIndexInputWrapper.java:40) at org.apache.lucene.store.MockIndexInputWrapper.clone(MockIndexInputWrapper.java:89) at org.apache.lucene.store.MockIndexInputWrapper.clone(MockIndexInputWrapper.java:29) at org.apache.lucene.codecs.blockterms.BlockTermsReader$FieldReader$SegmentTermsEnum.<init>(BlockTermsReader.java:331) at org.apache.lucene.codecs.blockterms.BlockTermsReader$FieldReader.iterator(BlockTermsReader.java:243) at org.apache.lucene.search.TermQuery$TermWeight.getTermsEnum(TermQuery.java:121) at org.apache.lucene.search.TermQuery$TermWeight.scorer(TermQuery.java:90) at org.apache.lucene.search.Weight.bulkScorer(Weight.java:147) at org.apache.lucene.search.BooleanWeight.optionalBulkScorer(BooleanWeight.java:157) at org.apache.lucene.search.BooleanWeight.booleanScorer(BooleanWeight.java:214) at org.apache.lucene.search.BooleanWeight.bulkScorer(BooleanWeight.java:283) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:655) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:462) at org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:215) at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1591) at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1408) at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:575) at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:544) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177) at org.apache.solr.core.SolrCore.execute(SolrCore.java:2473) at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:720) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:526) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:378) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:322) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:534) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 16) Thread[id=20408, name=searcherExecutor-5776-thread-1, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) 17) Thread[id=20543, name=qtp1812211277-20543, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest] at org.apache.lucene.codecs.blockterms.VariableGapTermsIndexReader$IndexEnum.<init>(VariableGapTermsIndexReader.java:97) at org.apache.lucene.codecs.blockterms.VariableGapTermsIndexReader.getFieldEnum(VariableGapTermsIndexReader.java:189) at org.apache.lucene.codecs.blockterms.BlockTermsReader$FieldReader$SegmentTermsEnum.<init>(BlockTermsReader.java:333) at org.apache.lucene.codecs.blockterms.BlockTermsReader$FieldReader.iterator(BlockTermsReader.java:243) at org.apache.lucene.search.TermQuery$TermWeight.getTermsEnum(TermQuery.java:121) at org.apache.lucene.search.TermQuery$TermWeight.scorer(TermQuery.java:90) at org.apache.lucene.search.Weight.bulkScorer(Weight.java:147) at org.apache.lucene.search.BooleanWeight.optionalBulkScorer(BooleanWeight.java:157) at org.apache.lucene.search.BooleanWeight.booleanScorer(BooleanWeight.java:214) at org.apache.lucene.search.BooleanWeight.bulkScorer(BooleanWeight.java:283) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:655) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:462) at org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:215) at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1591) at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1408) at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:575) at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:544) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177) at org.apache.solr.core.SolrCore.execute(SolrCore.java:2473) at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:720) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:526) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:378) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:322) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:534) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 18) Thread[id=20397, name=searcherExecutor-5781-thread-1, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) 19) Thread[id=20495, name=qtp423659769-20495, state=BLOCKED, group=TGRP-FullSolrCloudDistribCmdsTest] at org.apache.log4j.Category.callAppenders(Category.java:204) at org.apache.log4j.Category.forcedLog(Category.java:391) at org.apache.log4j.Category.log(Category.java:856) at org.slf4j.impl.Log4jLoggerAdapter.info(Log4jLoggerAdapter.java:304) at org.apache.solr.core.SolrCore.execute(SolrCore.java:2478) at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:720) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:526) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:378) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:322) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:534) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 20) Thread[id=20512, name=qtp150412892-20512, state=BLOCKED, group=TGRP-FullSolrCloudDistribCmdsTest] at org.apache.log4j.Category.callAppenders(Category.java:204) at org.apache.log4j.Category.forcedLog(Category.java:391) at org.apache.log4j.Category.log(Category.java:856) at org.slf4j.impl.Log4jLoggerAdapter.info(Log4jLoggerAdapter.java:304) at org.apache.solr.core.SolrCore.execute(SolrCore.java:2478) at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:720) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:526) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:378) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:322) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:534) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 20 threads leaked from SUITE scope at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest:
1) Thread[id=20390, name=searcherExecutor-5760-thread-1, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=20080, name=qtp1812211277-20080, state=BLOCKED, group=TGRP-FullSolrCloudDistribCmdsTest]
at org.apache.log4j.Category.callAppenders(Category.java:204)
at org.apache.log4j.Category.forcedLog(Category.java:391)
at org.apache.log4j.Category.log(Category.java:856)
at org.slf4j.impl.Log4jLoggerAdapter.info(Log4jLoggerAdapter.java:304)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2478)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:720)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:526)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:378)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:322)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=20402, name=searcherExecutor-5763-thread-1, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=20389, name=searcherExecutor-5761-thread-1, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=20111, name=qtp150412892-20111, state=BLOCKED, group=TGRP-FullSolrCloudDistribCmdsTest]
at org.apache.lucene.store.MockIndexInputWrapper.readByte(MockIndexInputWrapper.java:140)
at org.apache.lucene.codecs.compressing.LZ4.decompress(LZ4.java:110)
at org.apache.lucene.codecs.compressing.CompressionMode$4.decompress(CompressionMode.java:138)
at org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader$BlockState.document(CompressingStoredFieldsReader.java:555)
at org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.document(CompressingStoredFieldsReader.java:571)
at org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.visitDocument(CompressingStoredFieldsReader.java:578)
at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat$AssertingStoredFieldsReader.visitDocument(AssertingStoredFieldsFormat.java:73)
at org.apache.lucene.index.CodecReader.document(CodecReader.java:83)
at org.apache.lucene.index.FilterLeafReader.document(FilterLeafReader.java:341)
at org.apache.lucene.index.FilterLeafReader.document(FilterLeafReader.java:341)
at org.apache.lucene.index.BaseCompositeReader.document(BaseCompositeReader.java:118)
at org.apache.solr.search.SolrDocumentFetcher.doc(SolrDocumentFetcher.java:198)
at org.apache.solr.response.DocsStreamer.next(DocsStreamer.java:155)
at org.apache.solr.response.DocsStreamer.next(DocsStreamer.java:59)
at org.apache.solr.response.BinaryResponseWriter$Resolver.writeResultsBody(BinaryResponseWriter.java:126)
at org.apache.solr.response.BinaryResponseWriter$Resolver.writeResults(BinaryResponseWriter.java:145)
at org.apache.solr.response.BinaryResponseWriter$Resolver.resolve(BinaryResponseWriter.java:89)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:239)
at org.apache.solr.common.util.JavaBinCodec.writeNamedList(JavaBinCodec.java:223)
at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:330)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
at org.apache.solr.common.util.JavaBinCodec.marshal(JavaBinCodec.java:155)
at org.apache.solr.response.BinaryResponseWriter.write(BinaryResponseWriter.java:52)
at org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:49)
at org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:806)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:535)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:378)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:322)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
6) Thread[id=20398, name=searcherExecutor-5765-thread-1, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
7) Thread[id=20525, name=qtp150412892-20525, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
at java.io.FilterOutputStream.write(FilterOutputStream.java:125)
at org.apache.lucene.util.TestRuleLimitSysouts$DelegateStream.write(TestRuleLimitSysouts.java:128)
at java.io.PrintStream.write(PrintStream.java:480)
at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:282)
at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:135)
at java.io.OutputStreamWriter.write(OutputStreamWriter.java:220)
at java.io.Writer.write(Writer.java:157)
at org.apache.log4j.helpers.QuietWriter.write(QuietWriter.java:48)
at org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:310)
at org.apache.log4j.WriterAppender.append(WriterAppender.java:162)
at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
at org.apache.log4j.Category.callAppenders(Category.java:206)
at org.apache.log4j.Category.forcedLog(Category.java:391)
at org.apache.log4j.Category.log(Category.java:856)
at org.slf4j.impl.Log4jLoggerAdapter.info(Log4jLoggerAdapter.java:304)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2478)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:720)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:526)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:378)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:322)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
8) Thread[id=20514, name=qtp423659769-20514, state=BLOCKED, group=TGRP-FullSolrCloudDistribCmdsTest]
at org.apache.log4j.Category.callAppenders(Category.java:204)
at org.apache.log4j.Category.forcedLog(Category.java:391)
at org.apache.log4j.Category.log(Category.java:856)
at org.slf4j.impl.Log4jLoggerAdapter.info(Log4jLoggerAdapter.java:304)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2478)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:720)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:526)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:378)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:322)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
9) Thread[id=20489, name=qtp423659769-20489, state=BLOCKED, group=TGRP-FullSolrCloudDistribCmdsTest]
at org.apache.log4j.Category.callAppenders(Category.java:204)
at org.apache.log4j.Category.forcedLog(Category.java:391)
at org.apache.log4j.Category.log(Category.java:856)
at org.slf4j.impl.Log4jLoggerAdapter.info(Log4jLoggerAdapter.java:381)
at org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:810)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:535)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:378)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:322)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
10) Thread[id=20501, name=qtp1812211277-20501, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
at java.util.Arrays.copyOfRange(Arrays.java:3664)
at java.lang.String.<init>(String.java:207)
at java.lang.StringBuilder.toString(StringBuilder.java:407)
at org.apache.lucene.store.MockIndexInputWrapper.<init>(MockIndexInputWrapper.java:40)
at org.apache.lucene.store.MockIndexInputWrapper.clone(MockIndexInputWrapper.java:89)
at org.apache.lucene.store.MockIndexInputWrapper.clone(MockIndexInputWrapper.java:29)
at org.apache.lucene.codecs.blockterms.BlockTermsReader$FieldReader$SegmentTermsEnum.<init>(BlockTermsReader.java:331)
at org.apache.lucene.codecs.blockterms.BlockTermsReader$FieldReader.iterator(BlockTermsReader.java:243)
at org.apache.lucene.search.TermQuery$TermWeight.getTermsEnum(TermQuery.java:121)
at org.apache.lucene.search.TermQuery$TermWeight.scorer(TermQuery.java:90)
at org.apache.lucene.search.Weight.bulkScorer(Weight.java:147)
at org.apache.lucene.search.BooleanWeight.optionalBulkScorer(BooleanWeight.java:157)
at org.apache.lucene.search.BooleanWeight.booleanScorer(BooleanWeight.java:214)
at org.apache.lucene.search.BooleanWeight.bulkScorer(BooleanWeight.java:283)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:655)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:462)
at org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:215)
at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1591)
at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1408)
at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:575)
at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:544)
at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2473)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:720)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:526)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:378)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:322)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
11) Thread[id=20400, name=searcherExecutor-5773-thread-1, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
12) Thread[id=20401, name=searcherExecutor-5767-thread-1, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
13) Thread[id=20393, name=searcherExecutor-5769-thread-1, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
14) Thread[id=20410, name=searcherExecutor-5771-thread-1, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
15) Thread[id=20166, name=qtp2106581206-20166, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
at java.lang.String.valueOf(String.java:2994)
at java.lang.StringBuilder.append(StringBuilder.java:131)
at org.apache.lucene.store.MockIndexInputWrapper.<init>(MockIndexInputWrapper.java:40)
at org.apache.lucene.store.MockIndexInputWrapper.clone(MockIndexInputWrapper.java:89)
at org.apache.lucene.store.MockIndexInputWrapper.clone(MockIndexInputWrapper.java:29)
at org.apache.lucene.codecs.blockterms.BlockTermsReader$FieldReader$SegmentTermsEnum.<init>(BlockTermsReader.java:331)
at org.apache.lucene.codecs.blockterms.BlockTermsReader$FieldReader.iterator(BlockTermsReader.java:243)
at org.apache.lucene.search.TermQuery$TermWeight.getTermsEnum(TermQuery.java:121)
at org.apache.lucene.search.TermQuery$TermWeight.scorer(TermQuery.java:90)
at org.apache.lucene.search.Weight.bulkScorer(Weight.java:147)
at org.apache.lucene.search.BooleanWeight.optionalBulkScorer(BooleanWeight.java:157)
at org.apache.lucene.search.BooleanWeight.booleanScorer(BooleanWeight.java:214)
at org.apache.lucene.search.BooleanWeight.bulkScorer(BooleanWeight.java:283)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:655)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:462)
at org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:215)
at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1591)
at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1408)
at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:575)
at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:544)
at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2473)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:720)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:526)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:378)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:322)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
16) Thread[id=20408, name=searcherExecutor-5776-thread-1, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
17) Thread[id=20543, name=qtp1812211277-20543, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
at org.apache.lucene.codecs.blockterms.VariableGapTermsIndexReader$IndexEnum.<init>(VariableGapTermsIndexReader.java:97)
at org.apache.lucene.codecs.blockterms.VariableGapTermsIndexReader.getFieldEnum(VariableGapTermsIndexReader.java:189)
at org.apache.lucene.codecs.blockterms.BlockTermsReader$FieldReader$SegmentTermsEnum.<init>(BlockTermsReader.java:333)
at org.apache.lucene.codecs.blockterms.BlockTermsReader$FieldReader.iterator(BlockTermsReader.java:243)
at org.apache.lucene.search.TermQuery$TermWeight.getTermsEnum(TermQuery.java:121)
at org.apache.lucene.search.TermQuery$TermWeight.scorer(TermQuery.java:90)
at org.apache.lucene.search.Weight.bulkScorer(Weight.java:147)
at org.apache.lucene.search.BooleanWeight.optionalBulkScorer(BooleanWeight.java:157)
at org.apache.lucene.search.BooleanWeight.booleanScorer(BooleanWeight.java:214)
at org.apache.lucene.search.BooleanWeight.bulkScorer(BooleanWeight.java:283)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:655)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:462)
at org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:215)
at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1591)
at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1408)
at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:575)
at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:544)
at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2473)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:720)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:526)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:378)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:322)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
18) Thread[id=20397, name=searcherExecutor-5781-thread-1, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
19) Thread[id=20495, name=qtp423659769-20495, state=BLOCKED, group=TGRP-FullSolrCloudDistribCmdsTest]
at org.apache.log4j.Category.callAppenders(Category.java:204)
at org.apache.log4j.Category.forcedLog(Category.java:391)
at org.apache.log4j.Category.log(Category.java:856)
at org.slf4j.impl.Log4jLoggerAdapter.info(Log4jLoggerAdapter.java:304)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2478)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:720)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java
[...truncated too long message...]
.handle(GzipHandler.java:395)
[junit4] > at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] > at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] > at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] > at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] > at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
[junit4] > at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
[junit4] > at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 14) Thread[id=20495, name=qtp423659769-20495, state=BLOCKED, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4] > at org.apache.log4j.Category.callAppenders(Category.java:204)
[junit4] > at org.apache.log4j.Category.forcedLog(Category.java:391)
[junit4] > at org.apache.log4j.Category.log(Category.java:856)
[junit4] > at org.slf4j.impl.Log4jLoggerAdapter.info(Log4jLoggerAdapter.java:381)
[junit4] > at org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:810)
[junit4] > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:535)
[junit4] > at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:378)
[junit4] > at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:322)
[junit4] > at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
[junit4] > at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] > at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
[junit4] > at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] > at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] > at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] > at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] > at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] > at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] > at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] > at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395)
[junit4] > at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] > at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] > at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] > at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] > at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
[junit4] > at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
[junit4] > at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 15) Thread[id=20512, name=qtp150412892-20512, state=BLOCKED, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4] > at org.apache.log4j.Category.callAppenders(Category.java:204)
[junit4] > at org.apache.log4j.Category.forcedLog(Category.java:391)
[junit4] > at org.apache.log4j.Category.log(Category.java:856)
[junit4] > at org.slf4j.impl.Log4jLoggerAdapter.warn(Log4jLoggerAdapter.java:400)
[junit4] > at org.apache.solr.metrics.SolrMetricManager.closeReporters(SolrMetricManager.java:943)
[junit4] > at org.apache.solr.metrics.SolrCoreMetricManager.close(SolrCoreMetricManager.java:146)
[junit4] > at org.apache.solr.core.SolrCore.close(SolrCore.java:1581)
[junit4] > at org.apache.solr.servlet.HttpSolrCall.destroy(HttpSolrCall.java:579)
[junit4] > at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:391)
[junit4] > at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:322)
[junit4] > at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
[junit4] > at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] > at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
[junit4] > at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] > at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] > at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] > at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] > at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] > at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] > at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] > at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395)
[junit4] > at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] > at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] > at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] > at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] > at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
[junit4] > at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
[junit4] > at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([312D14671E6B3DDE]:0)
[junit4] Completed [505/727 (2!)] on J0 in 1699.93s, 1 test, 3 errors <<< FAILURES!
[...truncated 675 lines...]
[junit4] JVM J0: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/temp/junit4-J0-20170712_045056_5712924593271098990980.sysout
[junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
[junit4] java.lang.OutOfMemoryError: Java heap space
[junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/heapdumps/java_pid18990.hprof ...
[junit4] Heap dump file created [675978761 bytes in 17.155 secs]
[junit4] <<< JVM J0: EOF ----
[...truncated 7505 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/build.xml:818: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/build.xml:762: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid18990.hprof
Total time: 376 minutes 52 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 6 - Failure
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/6/
No tests ran.
Build Log:
[...truncated 79 lines...]
BUILD FAILED
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/build.xml:818: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/common-build.xml:422: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/common-build.xml:459: Ivy is not available
Total time: 0 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any