You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2018/03/05 04:36:46 UTC
[JENKINS] Lucene-Solr-Tests-7.x - Build # 481 - Unstable
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/481/
3 tests failed.
FAILED: org.apache.lucene.codecs.lucene54.TestLucene54DocValuesFormat.testSortedNumericsSingleValuedMissingVsStoredFields
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([C5389FDCE7D3ACCC]:0)
FAILED: junit.framework.TestSuite.org.apache.lucene.codecs.lucene54.TestLucene54DocValuesFormat
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([C5389FDCE7D3ACCC]:0)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost
Error Message:
/autoscaling/nodeAdded/127.0.0.1:10119_solr
Stack Trace:
java.util.NoSuchElementException: /autoscaling/nodeAdded/127.0.0.1:10119_solr
at __randomizedtesting.SeedInfo.seed([B4C91F39E209F767:BDCD1C761E392E1]:0)
at org.apache.solr.cloud.autoscaling.sim.SimDistribStateManager$Node.removeChild(SimDistribStateManager.java:163)
at org.apache.solr.cloud.autoscaling.sim.SimDistribStateManager$Node.removeEphemeralChildren(SimDistribStateManager.java:195)
at org.apache.solr.cloud.autoscaling.sim.SimDistribStateManager$Node.removeEphemeralChildren(SimDistribStateManager.java:197)
at org.apache.solr.cloud.autoscaling.sim.SimDistribStateManager$Node.removeEphemeralChildren(SimDistribStateManager.java:197)
at org.apache.solr.cloud.autoscaling.sim.SimClusterStateProvider.simRemoveNode(SimClusterStateProvider.java:253)
at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.simRemoveNode(SimCloudManager.java:273)
at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.simRemoveRandomNodes(SimCloudManager.java:293)
at org.apache.solr.cloud.autoscaling.sim.SimSolrCloudTestCase.setUp(SimSolrCloudTestCase.java:161)
at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:968)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 5027 lines...]
[junit4] Suite: org.apache.lucene.codecs.lucene54.TestLucene54DocValuesFormat
[junit4] IGNOR/A 0.00s J0 | TestLucene54DocValuesFormat.testTermsEnumRandomMany
[junit4] > Assumption #1: 'nightly' test group is disabled (@Nightly())
[junit4] IGNOR/A 0.00s J0 | TestLucene54DocValuesFormat.testSortedSetVariableLengthManyVsStoredFields
[junit4] > Assumption #1: 'nightly' test group is disabled (@Nightly())
[junit4] 2> Mar 04, 2018 7:39:29 PM com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
[junit4] 2> WARNING: Suite execution timed out: org.apache.lucene.codecs.lucene54.TestLucene54DocValuesFormat
[junit4] 2> 1) Thread[id=1, name=main, state=WAITING, group=main]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Thread.join(Thread.java:1252)
[junit4] 2> at java.lang.Thread.join(Thread.java:1326)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:636)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:493)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:251)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:368)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:13)
[junit4] 2> 2) Thread[id=17, name=SUITE-TestLucene54DocValuesFormat-seed#[C5389FDCE7D3ACCC], state=RUNNABLE, group=TGRP-TestLucene54DocValuesFormat]
[junit4] 2> at java.lang.Thread.getStackTrace(Thread.java:1559)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:696)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:693)
[junit4] 2> at java.security.AccessController.doPrivileged(Native Method)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:693)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:709)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:689)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:65)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:415)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:705)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:139)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:626)
[junit4] 2> 3) Thread[id=18, name=TEST-TestLucene54DocValuesFormat.testSortedNumericsSingleValuedMissingVsStoredFields-seed#[C5389FDCE7D3ACCC], state=TIMED_WAITING, group=TGRP-TestLucene54DocValuesFormat]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at org.apache.lucene.index.IndexWriter.doWait(IndexWriter.java:4679)
[junit4] 2> at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:2103)
[junit4] 2> at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:2031)
[junit4] 2> at org.apache.lucene.index.RandomIndexWriter.forceMerge(RandomIndexWriter.java:440)
[junit4] 2> at org.apache.lucene.index.BaseDocValuesFormatTestCase.doTestSortedNumericsVsStoredFields(BaseDocValuesFormatTestCase.java:1314)
[junit4] 2> at org.apache.lucene.index.BaseDocValuesFormatTestCase.testSortedNumericsSingleValuedMissingVsStoredFields(BaseDocValuesFormatTestCase.java:2248)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 4) Thread[id=9, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:50)
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestLucene54DocValuesFormat -Dtests.method=testSortedNumericsSingleValuedMissingVsStoredFields -Dtests.seed=C5389FDCE7D3ACCC -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=en-CA -Dtests.timezone=SystemV/PST8 -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 6982s J0 | TestLucene54DocValuesFormat.testSortedNumericsSingleValuedMissingVsStoredFields <<<
[junit4] > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
[junit4] > at __randomizedtesting.SeedInfo.seed([C5389FDCE7D3ACCC]:0)
[junit4] 2> Mar 04, 2018 7:39:29 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> Mar 04, 2018 7:39:49 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.codecs.lucene54.TestLucene54DocValuesFormat:
[junit4] 2> 1) Thread[id=18, name=TEST-TestLucene54DocValuesFormat.testSortedNumericsSingleValuedMissingVsStoredFields-seed#[C5389FDCE7D3ACCC], state=TIMED_WAITING, group=TGRP-TestLucene54DocValuesFormat]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at org.apache.lucene.index.IndexWriter.doWait(IndexWriter.java:4679)
[junit4] 2> at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:2103)
[junit4] 2> at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:2031)
[junit4] 2> at org.apache.lucene.index.RandomIndexWriter.forceMerge(RandomIndexWriter.java:440)
[junit4] 2> at org.apache.lucene.index.BaseDocValuesFormatTestCase.doTestSortedNumericsVsStoredFields(BaseDocValuesFormatTestCase.java:1314)
[junit4] 2> at org.apache.lucene.index.BaseDocValuesFormatTestCase.testSortedNumericsSingleValuedMissingVsStoredFields(BaseDocValuesFormatTestCase.java:2248)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Mar 04, 2018 7:39:49 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=18, name=TEST-TestLucene54DocValuesFormat.testSortedNumericsSingleValuedMissingVsStoredFields-seed#[C5389FDCE7D3ACCC], state=TIMED_WAITING, group=TGRP-TestLucene54DocValuesFormat]
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/backward-codecs/test/J0/temp/lucene.codecs.lucene54.TestLucene54DocValuesFormat_C5389FDCE7D3ACCC-001
[junit4] 2> Mar 05, 2018 3:39:49 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: All leaked threads terminated.
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=true): {fieldname=DFR I(F)Z(0.3), docId=DFR I(n)BZ(0.3), id=DFR GLZ(0.3)}, locale=en-CA, timezone=SystemV/PST8
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=312079360,total=353370112
[junit4] 2> NOTE: All tests run in this JVM: [TestLucene50SegmentInfoFormat, TestIndexWriterOnOldIndex, TestLegacyStringHelper, TestLucene54DocValuesFormat]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestLucene54DocValuesFormat -Dtests.seed=C5389FDCE7D3ACCC -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=en-CA -Dtests.timezone=SystemV/PST8 -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J0 | TestLucene54DocValuesFormat (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([C5389FDCE7D3ACCC]:0)
[junit4] Completed [9/9 (1!)] on J0 in 7220.44s, 91 tests, 2 errors, 2 skipped <<< FAILURES!
[...truncated 7280 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestLargeCluster_B4C91F39E209F767-001/init-core-data-001
[junit4] 2> 24169 INFO (SUITE-TestLargeCluster-seed#[B4C91F39E209F767]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 24170 INFO (SUITE-TestLargeCluster-seed#[B4C91F39E209F767]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 24170 INFO (SUITE-TestLargeCluster-seed#[B4C91F39E209F767]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 24278 INFO (TEST-TestLargeCluster.testBasic-seed#[B4C91F39E209F767]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testBasic
[junit4] 2> 24281 INFO (TEST-TestLargeCluster.testBasic-seed#[B4C91F39E209F767]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 24281 DEBUG (TEST-TestLargeCluster.testBasic-seed#[B4C91F39E209F767]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 24295 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 24290 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 24300 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 24355 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 24355 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 24356 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 24375 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 24377 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 24376 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 24409 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 24423 DEBUG (simCloudManagerPool-94-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 24424 DEBUG (simCloudManagerPool-94-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 24424 DEBUG (simCloudManagerPool-94-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 24424 DEBUG (simCloudManagerPool-94-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 24425 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 24427 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 24427 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 24429 DEBUG (ScheduledTrigger-95-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 24430 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 24444 DEBUG (simCloudManagerPool-94-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 24446 DEBUG (simCloudManagerPool-94-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 24446 DEBUG (simCloudManagerPool-94-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 24446 DEBUG (simCloudManagerPool-94-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 24447 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 24447 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 24447 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 24450 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 24450 DEBUG (ScheduledTrigger-95-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 24470 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 24471 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 24490 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 24491 DEBUG (ScheduledTrigger-95-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 24511 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 24512 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 24531 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 24532 DEBUG (ScheduledTrigger-95-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 24552 DEBUG (ScheduledTrigger-95-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 24552 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 24572 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 24573 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 24592 DEBUG (ScheduledTrigger-95-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 24593 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 24615 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 24615 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 24636 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 24636 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 24656 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 24656 DEBUG (ScheduledTrigger-95-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 24677 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 24677 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 24697 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 24697 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 24717 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 24717 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 24738 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 24738 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 24758 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 24758 DEBUG (ScheduledTrigger-95-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 24779 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 24779 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 24799 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 24799 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 24819 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 24819 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 24840 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 24840 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 24860 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 24860 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 24880 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 24880 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 24901 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 24901 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 24921 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 24921 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 24941 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 24942 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 24962 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 24962 DEBUG (ScheduledTrigger-95-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 24982 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 24982 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 25003 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 25003 DEBUG (ScheduledTrigger-95-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 25023 DEBUG (ScheduledTrigger-95-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 25023 DEBUG (ScheduledTrigger-95-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 25043 DEBUG (ScheduledTrigger-95-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 25043 DEBUG (ScheduledTrigger-95-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 25064 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 25064 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 25084 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 25084 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 25104 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 25104 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 25125 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 25125 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 25145 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 25145 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 25165 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 25165 DEBUG (ScheduledTrigger-95-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 25186 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 25186 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 25206 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 25206 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 25226 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 25226 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 25247 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 25247 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 25267 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 25267 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 25287 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 25287 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 25308 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 25308 DEBUG (ScheduledTrigger-95-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 25328 DEBUG (ScheduledTrigger-95-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 25328 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 25348 DEBUG (ScheduledTrigger-95-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 25348 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 25369 DEBUG (ScheduledTrigger-95-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 25369 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 25389 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 25389 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 25399 DEBUG (simCloudManagerPool-94-thread-81) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testBasic / shard4: {"core_node54":{
[junit4] 2> "node_name":"127.0.0.1:10044_solr",
[junit4] 2> "core":"testBasic_shard4_replica_t54",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"TLOG",
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "shard":"shard4",
[junit4] 2> "collection":"testBasic"}}
[junit4] 2> 25399 DEBUG (simCloudManagerPool-94-thread-80) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testBasic / shard3: {"core_node39":{
[junit4] 2> "node_name":"127.0.0.1:10049_solr",
[junit4] 2> "core":"testBasic_shard3_replica_t39",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"TLOG",
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"testBasic"}}
[junit4] 2> 25399 DEBUG (simCloudManagerPool-94-thread-79) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testBasic / shard2: {"core_node18":{
[junit4] 2> "node_name":"127.0.0.1:10041_solr",
[junit4] 2> "core":"testBasic_shard2_replica_n18",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"testBasic"}}
[junit4] 2> 25399 DEBUG (simCloudManagerPool-94-thread-82) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testBasic / shard5: {"core_node65":{
[junit4] 2> "node_name":"127.0.0.1:10047_solr",
[junit4] 2> "core":"testBasic_shard5_replica_n65",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "shard":"shard5",
[junit4] 2> "collection":"testBasic"}}
[junit4] 2> 25399 DEBUG (simCloudManagerPool-94-thread-83) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testBasic / shard1: {"core_node1":{
[junit4] 2> "node_name":"127.0.0.1:10095_solr",
[junit4] 2> "core":"testBasic_shard1_replica_n1",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testBasic"}}
[junit4] 2> 25405 INFO (TEST-TestLargeCluster.testBasic-seed#[B4C91F39E209F767]) [ ] o.a.s.c.a.s.TestLargeCluster Ready after 263ms
[junit4] 2> 25409 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 99
[junit4] 2> 25409 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10079_solr
[junit4] 2> 25410 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 99
[junit4] 2> 25410 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10079_solr
[junit4] 2> 25430 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 95
[junit4] 2> 25430 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10019_solr
[junit4] 2> 25430 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10089_solr
[junit4] 2> 25430 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10043_solr
[junit4] 2> 25430 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10088_solr
[junit4] 2> 25430 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 95
[junit4] 2> 25430 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10019_solr
[junit4] 2> 25430 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10089_solr
[junit4] 2> 25430 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10043_solr
[junit4] 2> 25430 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10088_solr
[junit4] 2> 25450 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 93
[junit4] 2> 25450 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10081_solr
[junit4] 2> 25450 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10052_solr
[junit4] 2> 25451 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 93
[junit4] 2> 25451 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10081_solr
[junit4] 2> 25451 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10052_solr
[junit4] 2> 25471 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 92
[junit4] 2> 25471 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10065_solr
[junit4] 2> 25471 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 92
[junit4] 2> 25471 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10065_solr
[junit4] 2> 25491 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 92
[junit4] 2> 25492 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 92
[junit4] 2> 25512 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 92
[junit4] 2> 25512 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 92
[junit4] 2> 25512 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10079_solr]
[junit4] 2> 25513 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"91787b30e9d7bT9vnt3vudkz6zdao3zusr2vzbb",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":2559146382499195,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[2559146382499195],
[junit4] 2> "nodeNames":["127.0.0.1:10079_solr"]}}
[junit4] 2> 25513 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_lost_trigger]
[junit4] 2> 25518 DEBUG (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"91787b30e9d7bT9vnt3vudkz6zdao3zusr2vzbb",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":2559146382499195,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[2559146382499195],
[junit4] 2> "_enqueue_time_":2559151582047345,
[junit4] 2> "nodeNames":["127.0.0.1:10079_solr"]}}
[junit4] 2> 25525 DEBUG (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ComputePlanAction -- processing event: {
[junit4] 2> "id":"91787b30e9d7bT9vnt3vudkz6zdao3zusr2vzbb",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":2559146382499195,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[2559146382499195],
[junit4] 2> "_enqueue_time_":2559151582047345,
[junit4] 2> "nodeNames":["127.0.0.1:10079_solr"]}} with context properties: {BEFORE_ACTION=[compute]}
[junit4] 2> 25656 INFO (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10072_solr&inPlaceMove=true&replica=core_node71
[junit4] 2> 25658 INFO (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 2559158791 sessionWrapper.createTime 2559153667323795, this.sessionWrapper.createTime 2559153667323795
[junit4] 2> 25660 DEBUG (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ExecutePlanAction -- processing event: {
[junit4] 2> "id":"91787b30e9d7bT9vnt3vudkz6zdao3zusr2vzbb",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":2559146382499195,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[2559146382499195],
[junit4] 2> "_enqueue_time_":2559151582047345,
[junit4] 2> "nodeNames":["127.0.0.1:10079_solr"]}} with context properties: {operations=[org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@1938bd3f], AFTER_ACTION=[compute], BEFORE_ACTION=[compute, execute]}
[junit4] 2> 25660 DEBUG (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10072_solr&inPlaceMove=true&replica=core_node71
[junit4] 2> 25792 DEBUG (simCloudManagerPool-94-thread-125) [ ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node77":{
[junit4] 2> "core":"testBasic_shard5_replica_p76",
[junit4] 2> "shard":"shard5",
[junit4] 2> "collection":"testBasic",
[junit4] 2> "node_name":"127.0.0.1:10072_solr",
[junit4] 2> "type":"PULL"}}
[junit4] 2> 25970 DEBUG (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 25970 DEBUG (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger after 100ms
[junit4] 2> 25970 DEBUG (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 22579 ms for event id=91787b30e9d7bT9vnt3vudkz6zdao3zusr2vzbb
[junit4] 2> 26070 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 92
[junit4] 2> 26070 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 92
[junit4] 2> 26070 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10052_solr, 127.0.0.1:10043_solr, 127.0.0.1:10089_solr, 127.0.0.1:10081_solr, 127.0.0.1:10065_solr, 127.0.0.1:10019_solr, 127.0.0.1:10088_solr]
[junit4] 2> 26070 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10052_solr, 127.0.0.1:10043_solr, 127.0.0.1:10079_solr, 127.0.0.1:10089_solr, 127.0.0.1:10081_solr, 127.0.0.1:10019_solr, 127.0.0.1:10088_solr]
[junit4] 2> 26070 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"917882c51fb3dT9vnt3vudkz6zdao3zusr2vzbf",
[junit4] 2> "source":".auto_add_replicas",
[junit4] 2> "eventTime":2559148416957245,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 2559148416957245,
[junit4] 2> 2559147392082745,
[junit4] 2> 2559146365218045,
[junit4] 2> 2559147391327495,
[junit4] 2> 2559148414857645,
[junit4] 2> 2559147390353795,
[junit4] 2> 2559147392836695],
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10052_solr",
[junit4] 2> "127.0.0.1:10043_solr",
[junit4] 2> "127.0.0.1:10079_solr",
[junit4] 2> "127.0.0.1:10089_solr",
[junit4] 2> "127.0.0.1:10081_solr",
[junit4] 2> "127.0.0.1:10019_solr",
[junit4] 2> "127.0.0.1:10088_solr"]}}
[junit4] 2> 26070 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"917882d2be177T9vnt3vudkz6zdao3zusr2vzbe",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":2559148431237495,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 2559148431237495,
[junit4] 2> 2559147404471145,
[junit4] 2> 2559147403701395,
[junit4] 2> 2559148430356745,
[junit4] 2> 2559149454371295,
[junit4] 2> 2559147402676545,
[junit4] 2> 2559147405236895],
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10052_solr",
[junit4] 2> "127.0.0.1:10043_solr",
[junit4] 2> "127.0.0.1:10089_solr",
[junit4] 2> "127.0.0.1:10081_solr",
[junit4] 2> "127.0.0.1:10065_solr",
[junit4] 2> "127.0.0.1:10019_solr",
[junit4] 2> "127.0.0.1:10088_solr"]}}
[junit4] 2> 26070 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_lost_trigger]
[junit4] 2> 26074 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger processor for lost nodes: [127.0.0.1:10052_solr, 127.0.0.1:10043_solr, 127.0.0.1:10089_solr, 127.0.0.1:10081_solr, 127.0.0.1:10065_solr, 127.0.0.1:10019_solr, 127.0.0.1:10088_solr] is not ready, will try later
[junit4] 2> 26087 DEBUG (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"917882c51fb3dT9vnt3vudkz6zdao3zusr2vzbf",
[junit4] 2> "source":".auto_add_replicas",
[junit4] 2> "eventTime":2559148416957245,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 2559148416957245,
[junit4] 2> 2559147392082745,
[junit4] 2> 2559146365218045,
[junit4] 2> 2559147391327495,
[junit4] 2> 2559148414857645,
[junit4] 2> 2559147390353795,
[junit4] 2> 2559147392836695],
[junit4] 2> "_enqueue_time_":2559179408077745,
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10052_solr",
[junit4] 2> "127.0.0.1:10043_solr",
[junit4] 2> "127.0.0.1:10079_solr",
[junit4] 2> "127.0.0.1:10089_solr",
[junit4] 2> "127.0.0.1:10081_solr",
[junit4] 2> "127.0.0.1:10019_solr",
[junit4] 2> "127.0.0.1:10088_solr"]}}
[junit4] 2> 26087 DEBUG (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ComputePlanAction -- processing event: {
[junit4] 2> "id":"917882c51fb3dT9vnt3vudkz6zdao3zusr2vzbf",
[junit4] 2> "source":".auto_add_replicas",
[junit4] 2> "eventTime":2559148416957245,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 2559148416957245,
[junit4] 2> 2559147392082745,
[junit4] 2> 2559146365218045,
[junit4] 2> 2559147391327495,
[junit4] 2> 2559148414857645,
[junit4] 2> 2559147390353795,
[junit4] 2> 2559147392836695],
[junit4] 2> "_enqueue_time_":2559179408077745,
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10052_solr",
[junit4] 2> "127.0.0.1:10043_solr",
[junit4] 2> "127.0.0.1:10079_solr",
[junit4] 2> "127.0.0.1:10089_solr",
[junit4] 2> "127.0.0.1:10081_solr",
[junit4] 2> "127.0.0.1:10019_solr",
[junit4] 2> "127.0.0.1:10088_solr"]}} with context properties: {BEFORE_ACTION=[auto_add_replicas_plan]}
[junit4] 2> 26089 INFO (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 2559180346 sessionWrapper.createTime 2559180318264395, this.sessionWrapper.createTime 2559180318264395
[junit4] 2> 26089 DEBUG (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ExecutePlanAction -- processing event: {
[junit4] 2> "id":"917882c51fb3dT9vnt3vudkz6zdao3zusr2vzbf",
[junit4] 2> "source":".auto_add_replicas",
[junit4] 2> "eventTime":2559148416957245,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 2559148416957245,
[junit4] 2> 2559147392082745,
[junit4] 2> 2559146365218045,
[junit4] 2> 2559147391327495,
[junit4] 2> 2559148414857645,
[junit4] 2> 2559147390353795,
[junit4] 2> 2559147392836695],
[junit4] 2> "_enqueue_time_":2559179408077745,
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10052_solr",
[junit4] 2> "127.0.0.1:10043_solr",
[junit4] 2> "127.0.0.1:10079_solr",
[junit4] 2> "127.0.0.1:10089_solr",
[junit4] 2> "127.0.0.1:10081_solr",
[junit4] 2> "127.0.0.1:10019_solr",
[junit4] 2> "127.0.0.1:10088_solr"]}} with context properties: {AFTER_ACTION=[auto_add_replicas_plan], BEFORE_ACTION=[auto_add_replicas_plan, execute_plan]}
[junit4] 2> 26089 INFO (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ExecutePlanAction No operations to execute for event: {
[junit4] 2> "id":"917882c51fb3dT9vnt3vudkz6zdao3zusr2vzbf",
[junit4] 2> "source":".auto_add_replicas",
[junit4] 2> "eventTime":2559148416957245,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 2559148416957245,
[junit4] 2> 2559147392082745,
[junit4] 2> 2559146365218045,
[junit4] 2> 2559147391327495,
[junit4] 2> 2559148414857645,
[junit4] 2> 2559147390353795,
[junit4] 2> 2559147392836695],
[junit4] 2> "_enqueue_time_":2559179408077745,
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10052_solr",
[junit4] 2> "127.0.0.1:10043_solr",
[junit4] 2> "127.0.0.1:10079_solr",
[junit4] 2> "127.0.0.1:10089_solr",
[junit4] 2> "127.0.0.1:10081_solr",
[junit4] 2> "127.0.0.1:10019_solr",
[junit4] 2> "127.0.0.1:10088_solr"]}}
[junit4] 2> 26091 DEBUG (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 26091 DEBUG (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger after 100ms
[junit4] 2> 26091 DEBUG (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 192 ms for event id=917882c51fb3dT9vnt3vudkz6zdao3zusr2vzbf
[junit4] 2> 26191 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 92
[junit4] 2> 26191 DEBUG (ScheduledTrigger-95-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 92
[junit4] 2> 26191 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10065_solr]
[junit4] 2> 26191 DEBUG (ScheduledTrigger-95-thread-2) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10052_solr, 127.0.0.1:10043_solr, 127.0.0.1:10089_solr, 127.0.0.1:10081_solr, 127.0.0.1:10065_solr, 127.0.0.1:10019_solr, 127.0.0.1:10088_solr]
[junit4] 2> 26191 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"917886930a207T9vnt3vudkz6zdao3zusr2vzbj",
[junit4] 2> "source":".auto_add_replicas",
[junit4] 2> "eventTime":2559149438181895,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[2559149438181895],
[junit4] 2> "nodeNames":["127.0.0.1:10065_solr"]}}
[junit4] 2> 26191 DEBUG (ScheduledTrigger-95-thread-2) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"917882d2be177T9vnt3vudkz6zdao3zusr2vzbk",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":2559148431237495,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 2559148431237495,
[junit4] 2> 2559147404471145,
[junit4] 2> 2559147403701395,
[junit4] 2> 2559148430356745,
[junit4] 2> 2559149454371295,
[junit4] 2> 2559147402676545,
[junit4] 2> 2559147405236895],
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10052_solr",
[junit4] 2> "127.0.0.1:10043_solr",
[junit4] 2> "127.0.0.1:10089_solr",
[junit4] 2> "127.0.0.1:10081_solr",
[junit4] 2> "127.0.0.1:10065_solr",
[junit4] 2> "127.0.0.1:10019_solr",
[junit4] 2> "127.0.0.1:10088_solr"]}}
[junit4] 2> 26191 DEBUG (ScheduledTrigger-95-thread-4) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_lost_trigger]
[junit4] 2> 26211 DEBUG (ScheduledTrigger-95-thread-2) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger processor for lost nodes: [127.0.0.1:10052_solr, 127.0.0.1:10043_solr, 127.0.0.1:10089_solr, 127.0.0.1:10081_solr, 127.0.0.1:10065_solr, 127.0.0.1:10019_solr, 127.0.0.1:10088_solr] is not ready, will try later
[junit4] 2> 26214 DEBUG (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"917886930a207T9vnt3vudkz6zdao3zusr2vzbj",
[junit4] 2> "source":".auto_add_replicas",
[junit4] 2> "eventTime":2559149438181895,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[2559149438181895],
[junit4] 2> "_enqueue_time_":2559185454096745,
[junit4] 2> "nodeNames":["127.0.0.1:10065_solr"]}}
[junit4] 2> 26214 DEBUG (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ComputePlanAction -- processing event: {
[junit4] 2> "id":"917886930a207T9vnt3vudkz6zdao3zusr2vzbj",
[junit4] 2> "source":".auto_add_replicas",
[junit4] 2> "eventTime":2559149438181895,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[2559149438181895],
[junit4] 2> "_enqueue_time_":2559185454096745,
[junit4] 2> "nodeNames":["127.0.0.1:10065_solr"]}} with context properties: {BEFORE_ACTION=[auto_add_replicas_plan]}
[junit4] 2> 26216 INFO (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 2559186697 sessionWrapper.createTime 2559186681946845, this.sessionWrapper.createTime 2559186681946845
[junit4] 2> 26216 DEBUG (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ExecutePlanAction -- processing event: {
[junit4] 2> "id":"917886930a207T9vnt3vudkz6zdao3zusr2vzbj",
[junit4] 2> "source":".auto_add_replicas",
[junit4] 2> "eventTime":2559149438181895,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[2559149438181895],
[junit4] 2> "_enqueue_time_":2559185454096745,
[junit4] 2> "nodeNames":["127.0.0.1:10065_solr"]}} with context properties: {AFTER_ACTION=[auto_add_replicas_plan], BEFORE_ACTION=[auto_add_replicas_plan, execute_plan]}
[junit4] 2> 26216 INFO (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ExecutePlanAction No operations to execute for event: {
[junit4] 2> "id":"917886930a207T9vnt3vudkz6zdao3zusr2vzbj",
[junit4] 2> "source":".auto_add_replicas",
[junit4] 2> "eventTime":2559149438181895,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[2559149438181895],
[junit4] 2> "_enqueue_time_":2559185454096745,
[junit4] 2> "nodeNames":["127.0.0.1:10065_solr"]}}
[junit4] 2> 26224 DEBUG (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 26224 DEBUG (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger after 100ms
[junit4] 2> 26224 DEBUG (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 505 ms for event id=917886930a207T9vnt3vudkz6zdao3zusr2vzbj
[junit4] 2> 26325 DEBUG (ScheduledTrigger-95-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 92
[junit4] 2> 26325 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 92
[junit4] 2> 26325 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10052_solr, 127.0.0.1:10043_solr, 127.0.0.1:10089_solr, 127.0.0.1:10081_solr, 127.0.0.1:10065_solr, 127.0.0.1:10019_solr, 127.0.0.1:10088_solr]
[junit4] 2> 26325 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"917882d2be177T9vnt3vudkz6zdao3zusr2vzbo",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":2559148431237495,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 2559148431237495,
[junit4] 2> 2559147404471145,
[junit4] 2> 2559147403701395,
[junit4] 2> 2559148430356745,
[junit4] 2> 2559149454371295,
[junit4] 2> 2559147402676545,
[junit4] 2> 2559147405236895],
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10052_solr",
[junit4] 2> "127.0.0.1:10043_solr",
[junit4] 2> "127.0.0.1:10089_solr",
[junit4] 2> "127.0.0.1:10081_solr",
[junit4] 2> "127.0.0.1:10065_solr",
[junit4] 2> "127.0.0.1:10019_solr",
[junit4] 2> "127.0.0.1:10088_solr"]}}
[junit4] 2> 26325 DEBUG (ScheduledTrigger-95-thread-3) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_lost_trigger]
[junit4] 2> 26326 DEBUG (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"917882d2be177T9vnt3vudkz6zdao3zusr2vzbo",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":2559148431237495,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 2559148431237495,
[junit4] 2> 2559147404471145,
[junit4] 2> 2559147403701395,
[junit4] 2> 2559148430356745,
[junit4] 2> 2559149454371295,
[junit4] 2> 2559147402676545,
[junit4] 2> 2559147405236895],
[junit4] 2> "_enqueue_time_":2559192137619095,
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10052_solr",
[junit4] 2> "127.0.0.1:10043_solr",
[junit4] 2> "127.0.0.1:10089_solr",
[junit4] 2> "127.0.0.1:10081_solr",
[junit4] 2> "127.0.0.1:10065_solr",
[junit4] 2> "127.0.0.1:10019_solr",
[junit4] 2> "127.0.0.1:10088_solr"]}}
[junit4] 2> 26326 DEBUG (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ComputePlanAction -- processing event: {
[junit4] 2> "id":"917882d2be177T9vnt3vudkz6zdao3zusr2vzbo",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":2559148431237495,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 2559148431237495,
[junit4] 2> 2559147404471145,
[junit4] 2> 2559147403701395,
[junit4] 2> 2559148430356745,
[junit4] 2> 2559149454371295,
[junit4] 2> 2559147402676545,
[junit4] 2> 2559147405236895],
[junit4] 2> "_enqueue_time_":2559192137619095,
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10052_solr",
[junit4] 2> "127.0.0.1:10043_solr",
[junit4] 2> "127.0.0.1:10089_solr",
[junit4] 2> "127.0.0.1:10081_solr",
[junit4] 2> "127.0.0.1:10065_solr",
[junit4] 2> "127.0.0.1:10019_solr",
[junit4] 2> "127.0.0.1:10088_solr"]}} with context properties: {BEFORE_ACTION=[compute]}
[junit4] 2> 26581 INFO (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10056_solr&inPlaceMove=true&replica=core_node40
[junit4] 2> 26673 INFO (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10059_solr&inPlaceMove=true&replica=core_node53
[junit4] 2> 26774 INFO (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10053_solr&inPlaceMove=true&replica=core_node25
[junit4] 2> 26860 INFO (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10037_solr&inPlaceMove=true&replica=core_node75
[junit4] 2> 26899 INFO (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10094_solr&inPlaceMove=true&replica=core_node15
[junit4] 2> 26925 INFO (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10091_solr&inPlaceMove=true&replica=core_node13
[junit4] 2> 26937 INFO (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10034_solr&inPlaceMove=true&replica=core_node46
[junit4] 2> 26937 INFO (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 2559222762 sessionWrapper.createTime 2559192271723145, this.sessionWrapper.createTime 2559192271723145
[junit4] 2> 26940 DEBUG (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ExecutePlanAction -- processing event: {
[junit4] 2> "id":"917882d2be177T9vnt3vudkz6zdao3zusr2vzbo",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":2559148431237495,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 2559148431237495,
[junit4] 2> 2559147404471145,
[junit4] 2> 2559147403701395,
[junit4] 2> 2559148430356745,
[junit4] 2> 2559149454371295,
[junit4] 2> 2559147402676545,
[junit4] 2> 2559147405236895],
[junit4] 2> "_enqueue_time_":2559192137619095,
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10052_solr",
[junit4] 2> "127.0.0.1:10043_solr",
[junit4] 2> "127.0.0.1:10089_solr",
[junit4] 2> "127.0.0.1:10081_solr",
[junit4] 2> "127.0.0.1:10065_solr",
[junit4] 2> "127.0.0.1:10019_solr",
[junit4] 2> "127.0.0.1:10088_solr"]}} with context properties: {operations=[org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@7d2b82e9, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@66618d34, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@21bc97fc, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@45b0b8a2, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@642d4637, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@ecf7fab, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@4b890af9], AFTER_ACTION=[compute], BEFORE_ACTION=[compute, execute]}
[junit4] 2> 26940 DEBUG (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10056_solr&inPlaceMove=true&replica=core_node40
[junit4] 2> 27042 DEBUG (simCloudManagerPool-94-thread-141) [ ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node79":{
[junit4] 2> "core":"testBasic_shard3_replica_t78",
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"testBasic",
[junit4] 2> "node_name":"127.0.0.1:10056_solr",
[junit4] 2> "type":"TLOG"}}
[junit4] 2> 27155 DEBUG (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10059_solr&inPlaceMove=true&replica=core_node53
[junit4] 2> 27256 DEBUG (simCloudManagerPool-94-thread-149) [ ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node81":{
[junit4] 2> "core":"testBasic_shard4_replica_t80",
[junit4] 2> "shard":"shard4",
[junit4] 2> "collection":"testBasic",
[junit4] 2> "node_name":"127.0.0.1:10059_solr",
[junit4] 2> "type":"TLOG"}}
[junit4] 2> 27368 DEBUG (AutoscalingActionExecutor-96-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10053_solr&inPlaceMove=true&replica=core_node25
[junit4] 2> 27469 DEBUG (simCloudManagerPool-94-thread-157) [ ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node83":{
[junit4] 2> "core":"testBasic_shard2_replica_t82",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"test
[...truncated too long message...]
> 41700 INFO (AutoscalingActionExecutor-99-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testNodeAdded&targetNode=127.0.0.1:10112_solr&inPlaceMove=true&replica=core_node32
[junit4] 2> 42530 INFO (AutoscalingActionExecutor-99-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testNodeAdded&targetNode=127.0.0.1:10111_solr&inPlaceMove=true&replica=core_node82
[junit4] 2> 43771 INFO (AutoscalingActionExecutor-99-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testNodeAdded&targetNode=127.0.0.1:10114_solr&inPlaceMove=true&replica=core_node130
[junit4] 2> 44615 INFO (AutoscalingActionExecutor-99-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testNodeAdded&targetNode=127.0.0.1:10117_solr&inPlaceMove=true&replica=core_node114
[junit4] 2> 45441 INFO (AutoscalingActionExecutor-99-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testNodeAdded&targetNode=127.0.0.1:10118_solr&inPlaceMove=true&replica=core_node96
[junit4] 2> 46178 INFO (AutoscalingActionExecutor-99-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testNodeAdded&targetNode=127.0.0.1:10118_solr&inPlaceMove=true&replica=core_node48
[junit4] 2> 46990 INFO (AutoscalingActionExecutor-99-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testNodeAdded&targetNode=127.0.0.1:10117_solr&inPlaceMove=true&replica=core_node63
[junit4] 2> 47701 INFO (AutoscalingActionExecutor-99-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testNodeAdded&targetNode=127.0.0.1:10114_solr&inPlaceMove=true&replica=core_node74
[junit4] 2> 48416 INFO (AutoscalingActionExecutor-99-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testNodeAdded&targetNode=127.0.0.1:10111_solr&inPlaceMove=true&replica=core_node71
[junit4] 2> 49258 INFO (AutoscalingActionExecutor-99-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testNodeAdded&targetNode=127.0.0.1:10112_solr&inPlaceMove=true&replica=core_node105
[junit4] 2> 50221 INFO (AutoscalingActionExecutor-99-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testNodeAdded&targetNode=127.0.0.1:10113_solr&inPlaceMove=true&replica=core_node62
[junit4] 2> 51026 INFO (AutoscalingActionExecutor-99-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testNodeAdded&targetNode=127.0.0.1:10115_solr&inPlaceMove=true&replica=core_node88
[junit4] 2> 51787 INFO (AutoscalingActionExecutor-99-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testNodeAdded&targetNode=127.0.0.1:10110_solr&inPlaceMove=true&replica=core_node66
[junit4] 2> 52516 INFO (AutoscalingActionExecutor-99-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testNodeAdded&targetNode=127.0.0.1:10116_solr&inPlaceMove=true&replica=core_node68
[junit4] 2> 54407 INFO (AutoscalingActionExecutor-99-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 2560596218 sessionWrapper.createTime 2559695691476695, this.sessionWrapper.createTime 2559695691476695
[junit4] 2> 54408 WARN (AutoscalingActionExecutor-99-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Exception executing actions
[junit4] 2> java.lang.Exception: Error executing action: execute for trigger event: {
[junit4] 2> "id":"917fba80fd141T9vnt3vudkz6zdao3zusr2vzc8",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":2559644414234945,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 2559644414234945,
[junit4] 2> 2559644419650395,
[junit4] 2> 2559644421144445,
[junit4] 2> 2559644421954795,
[junit4] 2> 2559644415406345,
[junit4] 2> 2559644420396295,
[junit4] 2> 2559644418083495,
[junit4] 2> 2559644422695945,
[junit4] 2> 2559644418896995],
[junit4] 2> "_enqueue_time_":2559695258264295,
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10118_solr",
[junit4] 2> "127.0.0.1:10112_solr",
[junit4] 2> "127.0.0.1:10115_solr",
[junit4] 2> "127.0.0.1:10110_solr",
[junit4] 2> "127.0.0.1:10117_solr",
[junit4] 2> "127.0.0.1:10113_solr",
[junit4] 2> "127.0.0.1:10114_solr",
[junit4] 2> "127.0.0.1:10116_solr",
[junit4] 2> "127.0.0.1:10111_solr"]}}
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:307)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: org.apache.solr.common.SolrException: Unexpected exception while processing event: {
[junit4] 2> "id":"917fba80fd141T9vnt3vudkz6zdao3zusr2vzc8",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":2559644414234945,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 2559644414234945,
[junit4] 2> 2559644419650395,
[junit4] 2> 2559644421144445,
[junit4] 2> 2559644421954795,
[junit4] 2> 2559644415406345,
[junit4] 2> 2559644420396295,
[junit4] 2> 2559644418083495,
[junit4] 2> 2559644422695945,
[junit4] 2> 2559644418896995],
[junit4] 2> "_enqueue_time_":2559695258264295,
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10118_solr",
[junit4] 2> "127.0.0.1:10112_solr",
[junit4] 2> "127.0.0.1:10115_solr",
[junit4] 2> "127.0.0.1:10110_solr",
[junit4] 2> "127.0.0.1:10117_solr",
[junit4] 2> "127.0.0.1:10113_solr",
[junit4] 2> "127.0.0.1:10114_solr",
[junit4] 2> "127.0.0.1:10116_solr",
[junit4] 2> "127.0.0.1:10111_solr"]}}
[junit4] 2> at org.apache.solr.cloud.autoscaling.ExecutePlanAction.process(ExecutePlanAction.java:120)
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:304)
[junit4] 2> ... 6 more
[junit4] 2> Caused by: org.apache.solr.common.SolrException: Unexpected exception executing operation: action=MOVEREPLICA&async=node_added_trigger/917fba80fd141T9vnt3vudkz6zdao3zusr2vzc8/0&waitForFinalState=true&collection=testNodeAdded&targetNode=127.0.0.1:10116_solr&inPlaceMove=true&replica=core_node120
[junit4] 2> at org.apache.solr.cloud.autoscaling.ExecutePlanAction.process(ExecutePlanAction.java:110)
[junit4] 2> ... 7 more
[junit4] 2> Caused by: java.io.IOException: java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$121/384094886@2b2d1f0c rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@516a15f3[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 369]
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:462)
[junit4] 2> at org.apache.solr.cloud.autoscaling.ExecutePlanAction.process(ExecutePlanAction.java:80)
[junit4] 2> ... 7 more
[junit4] 2> Caused by: java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$121/384094886@2b2d1f0c rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@516a15f3[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 369]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:173)
[junit4] 2> at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:134)
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.submit(SimCloudManager.java:382)
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:459)
[junit4] 2> ... 8 more
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=304, maxMBSortInHeap=5.275285212874322, sim=RandomSimilarity(queryNorm=false): {}, locale=ar-QA, timezone=America/Moncton
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=238156696,total=346030080
[junit4] 2> NOTE: All tests run in this JVM: [TestBM25SimilarityFactory, TestJettySolrRunner, TestSolrIndexConfig, DistributedFacetPivotLongTailTest, OverriddenZkACLAndCredentialsProvidersTest, TestLargeCluster]
[junit4] Completed [28/779 (1!)] on J1 in 30.45s, 4 tests, 1 error, 1 skipped <<< FAILURES!
[...truncated 50520 lines...]
[JENKINS] Lucene-Solr-Tests-7.x - Build # 485 - Still unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/485/
3 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode
Error Message:
no MOVEREPLICA ops?
Stack Trace:
java.lang.AssertionError: no MOVEREPLICA ops?
at __randomizedtesting.SeedInfo.seed([BA1E2D21F215A097:1DF130823D582F8F]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode(TestLargeCluster.java:262)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost
Error Message:
Stack Trace:
java.util.concurrent.TimeoutException
at __randomizedtesting.SeedInfo.seed([BA1E2D21F215A097:50BE3DF71FFC511]:0)
at org.apache.solr.cloud.autoscaling.sim.SimSolrCloudTestCase.waitForState(SimSolrCloudTestCase.java:261)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.doTestNodeLost(TestLargeCluster.java:456)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost(TestLargeCluster.java:298)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.client.solrj.io.graph.GraphTest
Error Message:
Could not load collection from ZK: collection1
Stack Trace:
org.apache.solr.common.SolrException: Could not load collection from ZK: collection1
at __randomizedtesting.SeedInfo.seed([147F30B2DE91AE68]:0)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1231)
at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:669)
at org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:148)
at org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:131)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:154)
at org.apache.solr.client.solrj.io.graph.GraphTest.setupCluster(GraphTest.java:63)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:874)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/state.json
at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1215)
at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:340)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:340)
at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1243)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1229)
... 29 more
Build Log:
[...truncated 12365 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestLargeCluster_BA1E2D21F215A097-001/init-core-data-001
[junit4] 2> 481609 WARN (SUITE-TestLargeCluster-seed#[BA1E2D21F215A097]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
[junit4] 2> 481609 INFO (SUITE-TestLargeCluster-seed#[BA1E2D21F215A097]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 481610 INFO (SUITE-TestLargeCluster-seed#[BA1E2D21F215A097]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 481644 INFO (SUITE-TestLargeCluster-seed#[BA1E2D21F215A097]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 481722 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 481722 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
[junit4] 2> 481722 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 481723 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
[junit4] 2> 481723 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
[junit4] 2> 481724 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 481724 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
[junit4] 2> 481747 DEBUG (ScheduledTrigger-523-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 481747 INFO (TEST-TestLargeCluster.testAddNode-seed#[BA1E2D21F215A097]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testAddNode
[junit4] 2> 481747 DEBUG (TEST-TestLargeCluster.testAddNode-seed#[BA1E2D21F215A097]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 481748 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 481766 INFO (TEST-TestLargeCluster.testAddNode-seed#[BA1E2D21F215A097]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 481766 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 481766 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 481766 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220)
[junit4] 2> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335)
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:163)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 481766 DEBUG (TEST-TestLargeCluster.testAddNode-seed#[BA1E2D21F215A097]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 481774 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 481774 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 481774 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 481775 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
[junit4] 2> 481775 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 481775 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 481775 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 481775 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 481795 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 481816 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 481836 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 481857 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 481887 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 481896 DEBUG (simCloudManagerPool-525-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 481896 DEBUG (simCloudManagerPool-525-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 481927 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 481940 DEBUG (simCloudManagerPool-525-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 481940 DEBUG (simCloudManagerPool-525-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 481940 DEBUG (simCloudManagerPool-525-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=5, actions=[{name=compute, class=org.apache.solr.cloud.autoscaling.ComputePlanAction}, {name=execute, class=org.apache.solr.cloud.autoscaling.ExecutePlanAction}], enabled=true}
[junit4] 2> 481940 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 481940 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 481941 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 481941 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 481947 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 481967 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482005 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482005 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482005 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482028 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482028 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482028 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482048 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482048 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482048 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482068 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482068 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482068 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482089 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482089 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482089 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482136 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482136 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482136 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482179 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482179 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482179 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482208 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482208 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482208 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482236 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482236 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482236 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482256 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482256 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482257 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482285 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482285 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482285 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482305 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482305 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482305 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482325 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482325 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482326 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482346 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482346 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482346 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482399 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482399 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482399 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482435 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482436 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482436 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482472 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482472 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482472 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482492 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482492 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482492 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482512 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482513 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482513 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482545 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482545 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482545 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482567 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482567 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482567 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482594 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482594 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482594 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482627 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482627 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482627 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482663 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482663 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482663 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482696 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482696 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482696 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482716 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482716 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482716 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482748 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482749 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482749 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482769 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482769 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482769 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482789 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482789 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482789 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482826 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482826 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482826 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482862 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482862 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482862 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482891 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482891 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482891 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482931 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482931 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482931 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482952 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482952 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482952 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 482980 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 482980 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 482980 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483000 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483000 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483000 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483021 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483021 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483021 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483041 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483041 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483041 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483061 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483061 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483061 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483081 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483081 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483082 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483102 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483102 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483102 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483122 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483122 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483122 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483151 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483151 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483151 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483171 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483171 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483171 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483191 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483191 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483191 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483213 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483213 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483213 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483233 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483233 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483233 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483253 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483253 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483254 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483274 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483274 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483274 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483294 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483294 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483294 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483314 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483314 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483314 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483335 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483335 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483335 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483355 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483355 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483355 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483375 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483375 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483375 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483396 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483396 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483396 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483416 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483416 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483416 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483436 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483436 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483436 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483456 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483457 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483457 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483477 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483477 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483477 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483497 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483497 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483497 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483517 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483517 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483518 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483549 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483549 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483549 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483569 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483569 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483569 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483589 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483589 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483590 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483614 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483614 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483614 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483650 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483650 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483650 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483670 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483670 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483671 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483691 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483691 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483691 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483711 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483711 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483711 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483731 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483731 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483732 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483764 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483764 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483764 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483784 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483784 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483784 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483817 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483817 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483817 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483837 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483837 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483837 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483857 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483857 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483857 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483886 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483886 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483886 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483906 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483906 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483906 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483926 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483926 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483926 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483946 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483946 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483947 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483967 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483967 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483967 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 483987 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 483987 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 483987 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484007 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484007 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484007 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484028 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484028 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484028 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484048 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484048 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484048 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484080 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484080 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484080 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484101 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484101 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484101 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484121 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484121 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484121 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484141 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484141 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484141 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484162 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484162 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484162 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484198 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484198 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484198 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484234 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484235 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484235 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484267 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484267 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484267 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484287 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484287 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484287 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484308 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484308 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484308 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484336 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484336 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484336 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484372 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484372 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484372 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484401 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484401 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484401 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484425 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484425 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484425 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484453 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484454 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484454 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484486 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484486 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484486 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484506 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484506 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484506 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484527 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484527 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484527 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484551 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484551 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484551 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484571 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484571 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484571 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484591 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484591 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484591 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484616 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484616 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484616 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484636 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484636 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484636 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484656 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484656 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484656 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484680 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484680 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484680 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484701 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484701 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484701 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484729 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484729 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484729 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484758 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484758 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484758 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484778 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484778 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484778 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484798 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484798 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484798 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484822 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484822 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484822 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484847 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484847 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484847 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484883 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484883 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484883 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484915 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484916 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484916 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484944 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484944 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484944 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484976 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484977 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484976 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 484997 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 484997 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 484997 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 485021 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 485021 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 485022 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 485042 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 485042 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 485042 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 485078 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 485078 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 485078 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 485098 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 485098 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 485098 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 485131 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 485131 DEBUG (ScheduledTrigger-526-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 485131 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 485152 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 485152 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 485152 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 485180 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 485180 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 485180 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 485200 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 485200 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 485200 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 485236 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 485236 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 485236 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 485256 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 485256 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 485257 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 485289 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 485289 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 485289 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 485322 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 485322 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 485322 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 485342 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 485342 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 485342 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 485362 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 485362 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 485362 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 485390 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 485390 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 485390 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 485411 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 485411 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 485411 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 485431 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 485431 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 485431 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 485451 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 485451 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 485451 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 485484 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 485484 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 485484 DEBUG (ScheduledTrigger-526-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 485504 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 485504 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 485504 DEBUG (ScheduledTrigger-526-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 485536 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 485536 DEBUG (ScheduledTrigger-526-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[...truncated too long message...]
onse ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 363727 INFO (jetty-closer-844-thread-2) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:45054_solr
[junit4] 2> 363744 INFO (coreCloseExecutor-655-thread-1) [n:127.0.0.1:43554_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=216363728
[junit4] 2> 363773 INFO (jetty-closer-844-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 363773 INFO (zkCallback-861-thread-3-processing-n:127.0.0.1:43554_solr) [n:127.0.0.1:43554_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 363955 INFO (zkCallback-861-thread-3-processing-n:127.0.0.1:43554_solr) [n:127.0.0.1:43554_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0])
[junit4] 2> 363962 INFO (zkCallback-861-thread-2-processing-n:127.0.0.1:43554_solr) [n:127.0.0.1:43554_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0])
[junit4] 2> 363962 INFO (zkCallback-855-thread-2-processing-n:127.0.0.1:45054_solr) [n:127.0.0.1:45054_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
[junit4] 2> 363962 INFO (zkCallback-855-thread-1-processing-n:127.0.0.1:45054_solr) [n:127.0.0.1:45054_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
[junit4] 2> 363999 INFO (zkCallback-855-thread-3-processing-n:127.0.0.1:45054_solr) [n:127.0.0.1:45054_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 363999 INFO (coreCloseExecutor-656-thread-1) [n:127.0.0.1:45054_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SolrCore [collection1_shard2_replica_n2] CLOSING SolrCore org.apache.solr.core.SolrCore@243f0a2d
[junit4] 2> 363999 INFO (coreCloseExecutor-656-thread-1) [n:127.0.0.1:45054_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1.shard2.replica_n2, tag=608111149
[junit4] 2> 363999 INFO (coreCloseExecutor-656-thread-1) [n:127.0.0.1:45054_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@223290ab: rootName = solr_45054, domain = solr.core.collection1.shard2.replica_n2, service url = null, agent id = null] for registry solr.core.collection1.shard2.replica_n2 / com.codahale.metrics.MetricRegistry@2aa5c7d6
[junit4] 2> 364177 INFO (coreCloseExecutor-656-thread-1) [n:127.0.0.1:45054_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard2.leader, tag=608111149
[junit4] 2> 364393 INFO (jetty-closer-844-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@48857fa0{/solr,null,UNAVAILABLE}
[junit4] 2> 364393 INFO (jetty-closer-844-thread-1) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 364655 INFO (jetty-closer-844-thread-2) [ ] o.a.s.c.Overseer Overseer (id=72242074886144004-127.0.0.1:45054_solr-n_0000000000) closing
[junit4] 2> 364665 INFO (OverseerStateUpdate-72242074886144004-127.0.0.1:45054_solr-n_0000000000) [n:127.0.0.1:45054_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:45054_solr
[junit4] 2> 364665 WARN (OverseerAutoScalingTriggerThread-72242074886144004-127.0.0.1:45054_solr-n_0000000000) [n:127.0.0.1:45054_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 364758 INFO (jetty-closer-844-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@205014df{/solr,null,UNAVAILABLE}
[junit4] 2> 364775 INFO (jetty-closer-844-thread-2) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 364783 ERROR (SUITE-GraphTest-seed#[147F30B2DE91AE68]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 364811 INFO (SUITE-GraphTest-seed#[147F30B2DE91AE68]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:34440 34440
[junit4] 2> 364943 INFO (Thread-497) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:34440 34440
[junit4] 2> 364944 WARN (Thread-497) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/aliases.json
[junit4] 2> 3 /solr/clusterprops.json
[junit4] 2> 2 /solr/collections/collection1/terms/shard2
[junit4] 2> 2 /solr/security.json
[junit4] 2> 2 /solr/collections/collection1/collectionprops.json
[junit4] 2> 2 /solr/configs/conf
[junit4] 2> 2 /solr/collections/collection1/terms/shard1
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 14 /solr/collections/collection1/state.json
[junit4] 2> 3 /solr/clusterstate.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/live_nodes
[junit4] 2> 3 /solr/collections
[junit4] 2>
[junit4] 2> 364944 INFO (SUITE-GraphTest-seed#[147F30B2DE91AE68]-worker) [ ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
[junit4] 2> Mar 08, 2018 2:52:54 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=756, maxMBSortInHeap=5.430889803048664, sim=RandomSimilarity(queryNorm=true): {}, locale=ar-SD, timezone=Europe/Isle_of_Man
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=260851184,total=354418688
[junit4] 2> NOTE: All tests run in this JVM: [FieldValueEvaluatorTest, GreaterThanEvaluatorTest, SquareRootEvaluatorTest, NaturalLogEvaluatorTest, TestXMLEscaping, TestNamedListCodec, TestClusteringResponse, LessThanEvaluatorTest, TestV1toV2ApiMapper, TestEmbeddedSolrServer, SchemaTest, SolrExampleStreamingTest, FieldAnalysisResponseTest, UsingSolrJRefGuideExamplesTest, NotEvaluatorTest, JdbcTest, LargeVolumeBinaryJettyTest, TestPolicy, TestPathTrie, AscEvaluatorTest, BasicHttpSolrClientTest, ConcurrentUpdateSolrClientBuilderTest, HttpSolrClientConPoolTest, HttpSolrClientSSLAuthConPoolTest, TestCloudSolrClientConnections, GraphTest]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=GraphTest -Dtests.seed=147F30B2DE91AE68 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=ar-SD -Dtests.timezone=Europe/Isle_of_Man -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.00s J0 | GraphTest (suite) <<<
[junit4] > Throwable #1: org.apache.solr.common.SolrException: Could not load collection from ZK: collection1
[junit4] > at __randomizedtesting.SeedInfo.seed([147F30B2DE91AE68]:0)
[junit4] > at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1231)
[junit4] > at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:669)
[junit4] > at org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:148)
[junit4] > at org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:131)
[junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:154)
[junit4] > at org.apache.solr.client.solrj.io.graph.GraphTest.setupCluster(GraphTest.java:63)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/state.json
[junit4] > at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
[junit4] > at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
[junit4] > at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1215)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:340)
[junit4] > at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:340)
[junit4] > at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1243)
[junit4] > at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1229)
[junit4] > ... 29 more
[junit4] Completed [103/146 (1!)] on J0 in 21.78s, 0 tests, 1 error <<< FAILURES!
[...truncated 48741 lines...]
[JENKINS] Lucene-Solr-Tests-7.x - Build # 484 - Failure
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/484/
All tests passed
Build Log:
[...truncated 60809 lines...]
-ecj-javadoc-lint-src:
[mkdir] Created dir: /tmp/ecj707217579
[ecj-lint] Compiling 1165 source files to /tmp/ecj707217579
[ecj-lint] invalid Class-Path header in manifest of jar file: /x1/jenkins/.ivy2/cache/org.restlet.jee/org.restlet/jars/org.restlet-2.3.0.jar
[ecj-lint] invalid Class-Path header in manifest of jar file: /x1/jenkins/.ivy2/cache/org.restlet.jee/org.restlet.ext.servlet/jars/org.restlet.ext.servlet-2.3.0.jar
[ecj-lint] ----------
[ecj-lint] 1. ERROR in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/handler/admin/CollectionsHandler.java (at line 31)
[ecj-lint] import java.util.OptionalLong;
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] The import java.util.OptionalLong is never used
[ecj-lint] ----------
[ecj-lint] 1 problem (1 error)
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/build.xml:618: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/build.xml:101: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build.xml:682: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/common-build.xml:2082: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/common-build.xml:2121: Compile failed; see the compiler error output for details.
Total time: 84 minutes 13 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-Tests-7.x - Build # 483 - Still unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/483/
1 tests failed.
FAILED: org.apache.solr.client.solrj.request.TestV1toV2ApiMapper.testCommands
Error Message:
null value is not valid for name
Stack Trace:
java.lang.RuntimeException: null value is not valid for name
at __randomizedtesting.SeedInfo.seed([34B160AEAE8568CF:158DA956C4BF88AB]:0)
at org.apache.solr.client.solrj.request.V1toV2ApiMapper$ActionInfo.lambda$null$2(V1toV2ApiMapper.java:108)
at org.apache.solr.common.MapWriter.toMap(MapWriter.java:42)
at org.apache.solr.common.MapWriter$1.put(MapWriter.java:45)
at org.apache.solr.client.solrj.request.V1toV2ApiMapper$ActionInfo.lambda$serializeToV2Format$3(V1toV2ApiMapper.java:98)
at org.apache.solr.common.MapWriter.toMap(MapWriter.java:42)
at org.apache.solr.client.solrj.request.V1toV2ApiMapper$ActionInfo.convert(V1toV2ApiMapper.java:87)
at org.apache.solr.client.solrj.request.V1toV2ApiMapper.convert(V1toV2ApiMapper.java:137)
at org.apache.solr.client.solrj.request.TestV1toV2ApiMapper.testCommands(TestV1toV2ApiMapper.java:55)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 15319 lines...]
[junit4] Suite: org.apache.solr.client.solrj.request.TestV1toV2ApiMapper
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestV1toV2ApiMapper -Dtests.method=testCommands -Dtests.seed=34B160AEAE8568CF -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=et-EE -Dtests.timezone=Africa/Porto-Novo -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 0.03s J0 | TestV1toV2ApiMapper.testCommands <<<
[junit4] > Throwable #1: java.lang.RuntimeException: null value is not valid for name
[junit4] > at __randomizedtesting.SeedInfo.seed([34B160AEAE8568CF:158DA956C4BF88AB]:0)
[junit4] > at org.apache.solr.client.solrj.request.V1toV2ApiMapper$ActionInfo.lambda$null$2(V1toV2ApiMapper.java:108)
[junit4] > at org.apache.solr.common.MapWriter.toMap(MapWriter.java:42)
[junit4] > at org.apache.solr.common.MapWriter$1.put(MapWriter.java:45)
[junit4] > at org.apache.solr.client.solrj.request.V1toV2ApiMapper$ActionInfo.lambda$serializeToV2Format$3(V1toV2ApiMapper.java:98)
[junit4] > at org.apache.solr.common.MapWriter.toMap(MapWriter.java:42)
[junit4] > at org.apache.solr.client.solrj.request.V1toV2ApiMapper$ActionInfo.convert(V1toV2ApiMapper.java:87)
[junit4] > at org.apache.solr.client.solrj.request.V1toV2ApiMapper.convert(V1toV2ApiMapper.java:137)
[junit4] > at org.apache.solr.client.solrj.request.TestV1toV2ApiMapper.testCommands(TestV1toV2ApiMapper.java:55)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: test params are: codec=HighCompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=17759, maxDocsPerChunk=190, blockSize=798), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=17759, blockSize=798)), sim=RandomSimilarity(queryNorm=false): {}, locale=et-EE, timezone=Africa/Porto-Novo
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=150580288,total=295174144
[junit4] 2> NOTE: All tests run in this JVM: [SolrSchemalessExampleTest, ReverseEvaluatorTest, NotEvaluatorTest, TestV1toV2ApiMapper]
[junit4] Completed [11/146 (1!)] on J0 in 0.05s, 1 test, 1 error <<< FAILURES!
[...truncated 49007 lines...]
[JENKINS] Lucene-Solr-Tests-7.x - Build # 482 - Failure
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/482/
3 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.analytics.legacy.facet.LegacyFieldFacetCloudTest
Error Message:
Could not find collection:collection1
Stack Trace:
java.lang.AssertionError: Could not find collection:collection1
at __randomizedtesting.SeedInfo.seed([306B9C495D2AE712]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:155)
at org.apache.solr.analytics.legacy.LegacyAbstractAnalyticsCloudTest.setupCollection(LegacyAbstractAnalyticsCloudTest.java:51)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:874)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.MoveReplicaHDFSTest.testFailedMove
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([DEE56711D71C82D8:7428B4E360CF5708]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertFalse(Assert.java:68)
at org.junit.Assert.assertFalse(Assert.java:79)
at org.apache.solr.cloud.MoveReplicaTest.testFailedMove(MoveReplicaTest.java:301)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.MoveReplicaHDFSTest.testNormalMove
Error Message:
Error from server at http://127.0.0.1:36687/solr: delete the collection time out:180s
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:36687/solr: delete the collection time out:180s
at __randomizedtesting.SeedInfo.seed([DEE56711D71C82D8:783DD0B652FD40C2]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.MiniSolrCloudCluster.deleteAllCollections(MiniSolrCloudCluster.java:451)
at org.apache.solr.cloud.MoveReplicaTest.beforeTest(MoveReplicaTest.java:95)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:968)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12386 lines...]
[junit4] Suite: org.apache.solr.cloud.MoveReplicaHDFSTest
[junit4] 2> 0 INFO (SUITE-MoveReplicaHDFSTest-seed#[DEE56711D71C82D8]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_DEE56711D71C82D8-001/init-core-data-001
[junit4] 2> 317 INFO (SUITE-MoveReplicaHDFSTest-seed#[DEE56711D71C82D8]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 478 INFO (SUITE-MoveReplicaHDFSTest-seed#[DEE56711D71C82D8]-worker) [ ] o.e.j.u.log Logging initialized @3652ms to org.eclipse.jetty.util.log.Slf4jLog
[junit4] 2> 499 INFO (SUITE-MoveReplicaHDFSTest-seed#[DEE56711D71C82D8]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 560 INFO (SUITE-MoveReplicaHDFSTest-seed#[DEE56711D71C82D8]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_DEE56711D71C82D8-001/tempDir-001
[junit4] 2> 593 INFO (SUITE-MoveReplicaHDFSTest-seed#[DEE56711D71C82D8]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 610 INFO (Thread-1) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 611 INFO (Thread-1) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 809 INFO (SUITE-MoveReplicaHDFSTest-seed#[DEE56711D71C82D8]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:53882
[junit4] 2> 820 ERROR (Thread-1) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 918 INFO (zkConnectionManagerCallback-4-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1379 INFO (jetty-launcher-1-thread-4) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T17:57:37-03:30, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1387 INFO (jetty-launcher-1-thread-3) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T17:57:37-03:30, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1396 INFO (jetty-launcher-1-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T17:57:37-03:30, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1404 INFO (jetty-launcher-1-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T17:57:37-03:30, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1422 INFO (jetty-launcher-1-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1422 INFO (jetty-launcher-1-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1427 INFO (jetty-launcher-1-thread-3) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 1448 INFO (jetty-launcher-1-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1454 INFO (jetty-launcher-1-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1454 INFO (jetty-launcher-1-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 1456 INFO (jetty-launcher-1-thread-4) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1457 INFO (jetty-launcher-1-thread-4) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1457 INFO (jetty-launcher-1-thread-4) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 1461 INFO (jetty-launcher-1-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1461 INFO (jetty-launcher-1-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1466 INFO (jetty-launcher-1-thread-1) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 1488 INFO (jetty-launcher-1-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4e69dbd4{/solr,null,AVAILABLE}
[junit4] 2> 1488 INFO (jetty-launcher-1-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1be83993{/solr,null,AVAILABLE}
[junit4] 2> 1488 INFO (jetty-launcher-1-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5220a4cd{/solr,null,AVAILABLE}
[junit4] 2> 1494 INFO (jetty-launcher-1-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1a4d7101{/solr,null,AVAILABLE}
[junit4] 2> 1512 INFO (jetty-launcher-1-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3aff783c{HTTP/1.1,[http/1.1]}{127.0.0.1:36687}
[junit4] 2> 1514 INFO (jetty-launcher-1-thread-3) [ ] o.e.j.s.Server Started @4698ms
[junit4] 2> 1515 INFO (jetty-launcher-1-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36687}
[junit4] 2> 1514 INFO (jetty-launcher-1-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7e4e6719{HTTP/1.1,[http/1.1]}{127.0.0.1:38227}
[junit4] 2> 1512 INFO (jetty-launcher-1-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3ac104e9{HTTP/1.1,[http/1.1]}{127.0.0.1:37852}
[junit4] 2> 1518 INFO (jetty-launcher-1-thread-4) [ ] o.e.j.s.Server Started @4701ms
[junit4] 2> 1518 INFO (jetty-launcher-1-thread-1) [ ] o.e.j.s.Server Started @4701ms
[junit4] 2> 1512 INFO (jetty-launcher-1-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4512317c{HTTP/1.1,[http/1.1]}{127.0.0.1:37469}
[junit4] 2> 1518 INFO (jetty-launcher-1-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37852}
[junit4] 2> 1518 INFO (jetty-launcher-1-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38227}
[junit4] 2> 1520 INFO (jetty-launcher-1-thread-2) [ ] o.e.j.s.Server Started @4702ms
[junit4] 2> 1520 INFO (jetty-launcher-1-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37469}
[junit4] 2> 1560 ERROR (jetty-launcher-1-thread-4) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1560 ERROR (jetty-launcher-1-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1560 ERROR (jetty-launcher-1-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1560 ERROR (jetty-launcher-1-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1561 INFO (jetty-launcher-1-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 1561 INFO (jetty-launcher-1-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 1566 INFO (jetty-launcher-1-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1560 INFO (jetty-launcher-1-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 1560 INFO (jetty-launcher-1-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 1566 INFO (jetty-launcher-1-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1566 INFO (jetty-launcher-1-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1566 INFO (jetty-launcher-1-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1567 INFO (jetty-launcher-1-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1566 INFO (jetty-launcher-1-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1566 INFO (jetty-launcher-1-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1567 INFO (jetty-launcher-1-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1586 INFO (jetty-launcher-1-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-03-05T20:44:56.841Z
[junit4] 2> 1586 INFO (jetty-launcher-1-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-03-05T20:44:56.843Z
[junit4] 2> 1586 INFO (jetty-launcher-1-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-03-05T20:44:56.843Z
[junit4] 2> 1586 INFO (jetty-launcher-1-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-03-05T20:44:56.842Z
[junit4] 2> 1616 INFO (jetty-launcher-1-thread-3) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
[junit4] 2> 1625 INFO (jetty-launcher-1-thread-4) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
[junit4] 2> 1642 INFO (zkConnectionManagerCallback-6-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1642 INFO (zkConnectionManagerCallback-8-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1652 INFO (zkConnectionManagerCallback-10-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1652 INFO (zkConnectionManagerCallback-12-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1653 INFO (jetty-launcher-1-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1653 INFO (jetty-launcher-1-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1659 INFO (jetty-launcher-1-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1660 INFO (jetty-launcher-1-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2287 INFO (jetty-launcher-1-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53882/solr
[junit4] 2> 2290 INFO (jetty-launcher-1-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53882/solr
[junit4] 2> 2291 INFO (jetty-launcher-1-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53882/solr
[junit4] 2> 2292 INFO (jetty-launcher-1-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53882/solr
[junit4] 2> 2313 INFO (zkConnectionManagerCallback-23-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2322 INFO (zkConnectionManagerCallback-24-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2322 INFO (zkConnectionManagerCallback-26-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2328 INFO (zkConnectionManagerCallback-28-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2354 INFO (zkConnectionManagerCallback-34-thread-1-processing-n:127.0.0.1:36687_solr) [n:127.0.0.1:36687_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2354 INFO (zkConnectionManagerCallback-32-thread-1-processing-n:127.0.0.1:37852_solr) [n:127.0.0.1:37852_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2354 INFO (zkConnectionManagerCallback-36-thread-1-processing-n:127.0.0.1:37469_solr) [n:127.0.0.1:37469_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2359 INFO (zkConnectionManagerCallback-31-thread-1-processing-n:127.0.0.1:38227_solr) [n:127.0.0.1:38227_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2957 INFO (jetty-launcher-1-thread-2) [n:127.0.0.1:37469_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2958 INFO (jetty-launcher-1-thread-4) [n:127.0.0.1:38227_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2962 INFO (jetty-launcher-1-thread-3) [n:127.0.0.1:36687_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2966 INFO (jetty-launcher-1-thread-2) [n:127.0.0.1:37469_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37469_solr
[junit4] 2> 2969 INFO (jetty-launcher-1-thread-1) [n:127.0.0.1:37852_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2975 INFO (jetty-launcher-1-thread-4) [n:127.0.0.1:38227_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38227_solr
[junit4] 2> 2979 INFO (jetty-launcher-1-thread-3) [n:127.0.0.1:36687_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36687_solr
[junit4] 2> 2987 INFO (jetty-launcher-1-thread-1) [n:127.0.0.1:37852_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37852_solr
[junit4] 2> 2987 INFO (jetty-launcher-1-thread-2) [n:127.0.0.1:37469_solr ] o.a.s.c.Overseer Overseer (id=73403349492432906-127.0.0.1:37469_solr-n_0000000000) starting
[junit4] 2> 3005 INFO (zkCallback-29-thread-1-processing-n:127.0.0.1:37852_solr) [n:127.0.0.1:37852_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 3006 INFO (zkCallback-33-thread-1-processing-n:127.0.0.1:36687_solr) [n:127.0.0.1:36687_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 3013 INFO (zkCallback-35-thread-1-processing-n:127.0.0.1:37469_solr) [n:127.0.0.1:37469_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 3013 INFO (zkCallback-30-thread-1-processing-n:127.0.0.1:38227_solr) [n:127.0.0.1:38227_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 3277 INFO (jetty-launcher-1-thread-2) [n:127.0.0.1:37469_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37469_solr
[junit4] 2> 3279 INFO (zkCallback-33-thread-1-processing-n:127.0.0.1:36687_solr) [n:127.0.0.1:36687_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 3279 INFO (zkCallback-30-thread-1-processing-n:127.0.0.1:38227_solr) [n:127.0.0.1:38227_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 3279 INFO (zkCallback-29-thread-1-processing-n:127.0.0.1:37852_solr) [n:127.0.0.1:37852_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 3286 INFO (zkCallback-35-thread-1-processing-n:127.0.0.1:37469_solr) [n:127.0.0.1:37469_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 4152 INFO (jetty-launcher-1-thread-3) [n:127.0.0.1:36687_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36687.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58ce73ae
[junit4] 2> 4156 INFO (jetty-launcher-1-thread-3) [n:127.0.0.1:36687_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36687.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58ce73ae
[junit4] 2> 4157 INFO (jetty-launcher-1-thread-3) [n:127.0.0.1:36687_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36687.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58ce73ae
[junit4] 2> 4185 INFO (jetty-launcher-1-thread-2) [n:127.0.0.1:37469_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37469.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58ce73ae
[junit4] 2> 4186 INFO (jetty-launcher-1-thread-2) [n:127.0.0.1:37469_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37469.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58ce73ae
[junit4] 2> 4186 INFO (jetty-launcher-1-thread-2) [n:127.0.0.1:37469_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37469.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58ce73ae
[junit4] 2> 4179 INFO (jetty-launcher-1-thread-4) [n:127.0.0.1:38227_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38227.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58ce73ae
[junit4] 2> 4188 INFO (jetty-launcher-1-thread-4) [n:127.0.0.1:38227_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38227.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58ce73ae
[junit4] 2> 4188 INFO (jetty-launcher-1-thread-4) [n:127.0.0.1:38227_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38227.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58ce73ae
[junit4] 2> 4218 INFO (jetty-launcher-1-thread-1) [n:127.0.0.1:37852_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37852.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58ce73ae
[junit4] 2> 4218 INFO (jetty-launcher-1-thread-1) [n:127.0.0.1:37852_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37852.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58ce73ae
[junit4] 2> 4219 INFO (jetty-launcher-1-thread-1) [n:127.0.0.1:37852_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37852.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58ce73ae
[junit4] 2> 4256 INFO (jetty-launcher-1-thread-2) [n:127.0.0.1:37469_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_DEE56711D71C82D8-001/tempDir-001/node4/.
[junit4] 2> 4259 INFO (jetty-launcher-1-thread-1) [n:127.0.0.1:37852_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_DEE56711D71C82D8-001/tempDir-001/node1/.
[junit4] 2> 4266 INFO (jetty-launcher-1-thread-4) [n:127.0.0.1:38227_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_DEE56711D71C82D8-001/tempDir-001/node2/.
[junit4] 2> 4286 INFO (jetty-launcher-1-thread-3) [n:127.0.0.1:36687_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_DEE56711D71C82D8-001/tempDir-001/node3/.
[junit4] 2> 5138 INFO (zkConnectionManagerCallback-46-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5172 INFO (zkConnectionManagerCallback-50-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5175 INFO (SUITE-MoveReplicaHDFSTest-seed#[DEE56711D71C82D8]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 5177 INFO (SUITE-MoveReplicaHDFSTest-seed#[DEE56711D71C82D8]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:53882/solr ready
[junit4] 2> 5750 INFO (qtp692735685-44) [n:127.0.0.1:38227_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 5835 INFO (qtp692735685-44) [n:127.0.0.1:38227_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=187
[junit4] 2> 7135 WARN (SUITE-MoveReplicaHDFSTest-seed#[DEE56711D71C82D8]-worker) [ ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 7778 INFO (OverseerCollectionConfigSetProcessor-73403349492432906-127.0.0.1:37469_solr-n_0000000000) [n:127.0.0.1:37469_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 8704 WARN (SUITE-MoveReplicaHDFSTest-seed#[DEE56711D71C82D8]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 8967 INFO (SUITE-MoveReplicaHDFSTest-seed#[DEE56711D71C82D8]-worker) [ ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
[junit4] 2> 8996 WARN (SUITE-MoveReplicaHDFSTest-seed#[DEE56711D71C82D8]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 9449 INFO (SUITE-MoveReplicaHDFSTest-seed#[DEE56711D71C82D8]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 9514 INFO (SUITE-MoveReplicaHDFSTest-seed#[DEE56711D71C82D8]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_40864_hdfs____mw7lic/webapp
[junit4] 2> 10808 INFO (SUITE-MoveReplicaHDFSTest-seed#[DEE56711D71C82D8]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40864
[junit4] 2> 11972 WARN (SUITE-MoveReplicaHDFSTest-seed#[DEE56711D71C82D8]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 11978 INFO (SUITE-MoveReplicaHDFSTest-seed#[DEE56711D71C82D8]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 12009 INFO (SUITE-MoveReplicaHDFSTest-seed#[DEE56711D71C82D8]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_47748_datanode____.eqcxa9/webapp
[junit4] 2> 12601 INFO (SUITE-MoveReplicaHDFSTest-seed#[DEE56711D71C82D8]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:47748
[junit4] 2> 13378 WARN (SUITE-MoveReplicaHDFSTest-seed#[DEE56711D71C82D8]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 13380 INFO (SUITE-MoveReplicaHDFSTest-seed#[DEE56711D71C82D8]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 13404 INFO (SUITE-MoveReplicaHDFSTest-seed#[DEE56711D71C82D8]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_59718_datanode____fufsdd/webapp
[junit4] 2> 14149 INFO (SUITE-MoveReplicaHDFSTest-seed#[DEE56711D71C82D8]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:59718
[junit4] 2> 14508 ERROR (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_DEE56711D71C82D8-001/tempDir-002/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_DEE56711D71C82D8-001/tempDir-002/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:53179) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 14547 ERROR (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_DEE56711D71C82D8-001/tempDir-002/hdfsBaseDir/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_DEE56711D71C82D8-001/tempDir-002/hdfsBaseDir/data/data4/]] heartbeating to localhost/127.0.0.1:53179) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 14828 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x48f41f163ec6dc: from storage DS-bb38408c-b6aa-4f9c-913c-a617542ac63e node DatanodeRegistration(127.0.0.1:43829, datanodeUuid=e3955fe5-61c9-4705-b600-88c0e425407d, infoPort=53962, infoSecurePort=0, ipcPort=36499, storageInfo=lv=-56;cid=testClusterID;nsid=1798464040;c=0), blocks: 0, hasStaleStorage: true, processing time: 3 msecs
[junit4] 2> 14841 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x48f41f1631f798: from storage DS-7110f661-e4a3-4b6a-a554-e03695532630 node DatanodeRegistration(127.0.0.1:52789, datanodeUuid=09f435c9-ab86-4c51-8a1a-2b5c3cfb7db8, infoPort=54564, infoSecurePort=0, ipcPort=43492, storageInfo=lv=-56;cid=testClusterID;nsid=1798464040;c=0), blocks: 0, hasStaleStorage: true, processing time: 1 msecs
[junit4] 2> 14841 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x48f41f163ec6dc: from storage DS-1946115b-85d9-4d58-9e2c-90a734f25961 node DatanodeRegistration(127.0.0.1:43829, datanodeUuid=e3955fe5-61c9-4705-b600-88c0e425407d, infoPort=53962, infoSecurePort=0, ipcPort=36499, storageInfo=lv=-56;cid=testClusterID;nsid=1798464040;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 14845 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x48f41f1631f798: from storage DS-1b15baba-f70c-47bf-a4ef-819971e60443 node DatanodeRegistration(127.0.0.1:52789, datanodeUuid=09f435c9-ab86-4c51-8a1a-2b5c3cfb7db8, infoPort=54564, infoSecurePort=0, ipcPort=43492, storageInfo=lv=-56;cid=testClusterID;nsid=1798464040;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] IGNOR/A 0.01s J2 | MoveReplicaHDFSTest.testNormalFailedMove
[junit4] > Assumption #1: 'badapple' test group is disabled (@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] 2> 15581 INFO (TEST-MoveReplicaHDFSTest.test-seed#[DEE56711D71C82D8]) [ ] o.a.s.SolrTestCaseJ4 ###Starting test
[junit4] 2> 15587 INFO (TEST-MoveReplicaHDFSTest.test-seed#[DEE56711D71C82D8]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 15592 INFO (TEST-MoveReplicaHDFSTest.test-seed#[DEE56711D71C82D8]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T17:57:37-03:30, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 15610 INFO (TEST-MoveReplicaHDFSTest.test-seed#[DEE56711D71C82D8]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 15610 INFO (TEST-MoveReplicaHDFSTest.test-seed#[DEE56711D71C82D8]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 15611 INFO (TEST-MoveReplicaHDFSTest.test-seed#[DEE56711D71C82D8]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 15619 INFO (TEST-MoveReplicaHDFSTest.test-seed#[DEE56711D71C82D8]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7f977358{/solr,null,AVAILABLE}
[junit4] 2> 15620 INFO (TEST-MoveReplicaHDFSTest.test-seed#[DEE56711D71C82D8]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7b5f3c7d{HTTP/1.1,[http/1.1]}{127.0.0.1:49274}
[junit4] 2> 15620 INFO (TEST-MoveReplicaHDFSTest.test-seed#[DEE56711D71C82D8]) [ ] o.e.j.s.Server Started @18803ms
[junit4] 2> 15620 INFO (TEST-MoveReplicaHDFSTest.test-seed#[DEE56711D71C82D8]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=49274}
[junit4] 2> 15620 ERROR (TEST-MoveReplicaHDFSTest.test-seed#[DEE56711D71C82D8]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 15620 INFO (TEST-MoveReplicaHDFSTest.test-seed#[DEE56711D71C82D8]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 15621 INFO (TEST-MoveReplicaHDFSTest.test-seed#[DEE56711D71C82D8]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 15621 INFO (TEST-MoveReplicaHDFSTest.test-seed#[DEE56711D71C82D8]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 15621 INFO (TEST-MoveReplicaHDFSTest.test-seed#[DEE56711D71C82D8]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-03-05T20:45:10.894Z
[junit4] 2> 15660 INFO (zkConnectionManagerCallback-53-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 15664 INFO (TEST-MoveReplicaHDFSTest.test-seed#[DEE56711D71C82D8]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 15703 INFO (TEST-MoveReplicaHDFSTest.test-seed#[DEE56711D71C82D8]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53882/solr
[junit4] 2> 15718 INFO (zkConnectionManagerCallback-57-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 15771 INFO (zkConnectionManagerCallback-59-thread-1-processing-n:127.0.0.1:49274_solr) [n:127.0.0.1:49274_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 15792 INFO (TEST-MoveReplicaHDFSTest.test-seed#[DEE56711D71C82D8]) [n:127.0.0.1:49274_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 15796 INFO (TEST-MoveReplicaHDFSTest.test-seed#[DEE56711D71C82D8]) [n:127.0.0.1:49274_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 15800 INFO (TEST-MoveReplicaHDFSTest.test-seed#[DEE56711D71C82D8]) [n:127.0.0.1:49274_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49274_solr
[junit4] 2> 15805 INFO (zkCallback-29-thread-1-processing-n:127.0.0.1:37852_solr) [n:127.0.0.1:37852_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 15805 INFO (zkCallback-35-thread-1-processing-n:127.0.0.1:37469_solr) [n:127.0.0.1:37469_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 15805 INFO (zkCallback-30-thread-1-processing-n:127.0.0.1:38227_solr) [n:127.0.0.1:38227_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 15810 INFO (zkCallback-33-thread-1-processing-n:127.0.0.1:36687_solr) [n:127.0.0.1:36687_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 15814 INFO (zkCallback-49-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 15814 INFO (zkCallback-49-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 15818 INFO (zkCallback-58-thread-1-processing-n:127.0.0.1:49274_solr) [n:127.0.0.1:49274_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 16113 INFO (TEST-MoveReplicaHDFSTest.test-seed#[DEE56711D71C82D8]) [n:127.0.0.1:49274_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49274.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58ce73ae
[junit4] 2> 16158 INFO (TEST-MoveReplicaHDFSTest.test-seed#[DEE56711D71C82D8]) [n:127.0.0.1:49274_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49274.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58ce73ae
[junit4] 2> 16159 INFO (TEST-MoveReplicaHDFSTest.test-seed#[DEE56711D71C82D8]) [n:127.0.0.1:49274_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49274.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58ce73ae
[junit4] 2> 16171 INFO (TEST-MoveReplicaHDFSTest.test-seed#[DEE56711D71C82D8]) [n:127.0.0.1:49274_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_DEE56711D71C82D8-001/tempDir-001/node5/.
[junit4] 2> 16562 INFO (zkConnectionManagerCallback-63-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 16701 INFO (TEST-MoveReplicaHDFSTest.test-seed#[DEE56711D71C82D8]) [ ] o.a.s.c.MoveReplicaTest total_jettys: 5
[junit4] 2> 16739 INFO (qtp1578008132-29) [n:127.0.0.1:36687_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf1&maxShardsPerNode=2&autoAddReplicas=false&name=MoveReplicaHDFSTest_coll_true&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 16758 INFO (OverseerThreadFactory-20-thread-2-processing-n:127.0.0.1:37469_solr) [n:127.0.0.1:37469_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection MoveReplicaHDFSTest_coll_true
[junit4] 2> 16920 INFO (OverseerStateUpdate-73403349492432906-127.0.0.1:37469_solr-n_0000000000) [n:127.0.0.1:37469_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"MoveReplicaHDFSTest_coll_true",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"MoveReplicaHDFSTest_coll_true_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:36687/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 16928 INFO (OverseerStateUpdate-73403349492432906-127.0.0.1:37469_solr-n_0000000000) [n:127.0.0.1:37469_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"MoveReplicaHDFSTest_coll_true",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"MoveReplicaHDFSTest_coll_true_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:37469/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 16944 INFO (OverseerStateUpdate-73403349492432906-127.0.0.1:37469_solr-n_0000000000) [n:127.0.0.1:37469_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"MoveReplicaHDFSTest_coll_true",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"MoveReplicaHDFSTest_coll_true_shard2_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:38227/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 16950 INFO (OverseerStateUpdate-73403349492432906-127.0.0.1:37469_solr-n_0000000000) [n:127.0.0.1:37469_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"MoveReplicaHDFSTest_coll_true",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"MoveReplicaHDFSTest_coll_true_shard2_replica_n6",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:37852/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 17231 INFO (qtp2120345233-37) [n:127.0.0.1:37469_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf1&newCollection=true&name=MoveReplicaHDFSTest_coll_true_shard1_replica_n2&action=CREATE&numShards=2&collection=MoveReplicaHDFSTest_coll_true&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 17232 INFO (qtp731528774-58) [n:127.0.0.1:37852_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf1&newCollection=true&name=MoveReplicaHDFSTest_coll_true_shard2_replica_n6&action=CREATE&numShards=2&collection=MoveReplicaHDFSTest_coll_true&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 17240 INFO (qtp1578008132-34) [n:127.0.0.1:36687_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf1&newCollection=true&name=MoveReplicaHDFSTest_coll_true_shard1_replica_n1&action=CREATE&numShards=2&collection=MoveReplicaHDFSTest_coll_true&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 17241 INFO (qtp2120345233-37) [n:127.0.0.1:37469_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 17243 INFO (qtp692735685-52) [n:127.0.0.1:38227_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf1&newCollection=true&name=MoveReplicaHDFSTest_coll_true_shard2_replica_n4&action=CREATE&numShards=2&collection=MoveReplicaHDFSTest_coll_true&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 17243 INFO (qtp1578008132-34) [n:127.0.0.1:36687_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 17243 INFO (qtp731528774-58) [n:127.0.0.1:37852_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 17243 INFO (qtp692735685-52) [n:127.0.0.1:38227_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 17381 INFO (zkCallback-29-thread-1-processing-n:127.0.0.1:37852_solr) [n:127.0.0.1:37852_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_coll_true/state.json] for collection [MoveReplicaHDFSTest_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 17381 INFO (zkCallback-30-thread-1-processing-n:127.0.0.1:38227_solr) [n:127.0.0.1:38227_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_coll_true/state.json] for collection [MoveReplicaHDFSTest_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 17381 INFO (zkCallback-33-thread-1-processing-n:127.0.0.1:36687_solr) [n:127.0.0.1:36687_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_coll_true/state.json] for collection [MoveReplicaHDFSTest_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 17383 INFO (zkCallback-35-thread-1-processing-n:127.0.0.1:37469_solr) [n:127.0.0.1:37469_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_coll_true/state.json] for collection [MoveReplicaHDFSTest_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 17384 INFO (zkCallback-29-thread-2-processing-n:127.0.0.1:37852_solr) [n:127.0.0.1:37852_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_coll_true/state.json] for collection [MoveReplicaHDFSTest_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 17384 INFO (zkCallback-35-thread-2-processing-n:127.0.0.1:37469_solr) [n:127.0.0.1:37469_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_coll_true/state.json] for collection [MoveReplicaHDFSTest_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 17384 INFO (zkCallback-30-thread-2-processing-n:127.0.0.1:38227_solr) [n:127.0.0.1:38227_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_coll_true/state.json] for collection [MoveReplicaHDFSTest_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 17385 INFO (zkCallback-33-thread-2-processing-n:127.0.0.1:36687_solr) [n:127.0.0.1:36687_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_coll_true/state.json] for collection [MoveReplicaHDFSTest_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 18335 INFO (qtp2120345233-37) [n:127.0.0.1:37469_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 18355 INFO (qtp692735685-52) [n:127.0.0.1:38227_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 18380 INFO (qtp731528774-58) [n:127.0.0.1:37852_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 18381 INFO (qtp1578008132-34) [n:127.0.0.1:36687_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 18465 INFO (qtp692735685-52) [n:127.0.0.1:38227_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.s.IndexSchema [MoveReplicaHDFSTest_coll_true_shard2_replica_n4] Schema name=minimal
[junit4] 2> 18468 INFO (qtp1578008132-34) [n:127.0.0.1:36687_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.s.IndexSchema [MoveReplicaHDFSTest_coll_true_shard1_replica_n1] Schema name=minimal
[junit4] 2> 18468 INFO (qtp2120345233-37) [n:127.0.0.1:37469_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.s.IndexSchema [MoveReplicaHDFSTest_coll_true_shard1_replica_n2] Schema name=minimal
[junit4] 2> 18481 INFO (qtp731528774-58) [n:127.0.0.1:37852_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.s.IndexSchema [MoveReplicaHDFSTest_coll_true_shard2_replica_n6] Schema name=minimal
[junit4] 2> 18587 INFO (qtp2120345233-37) [n:127.0.0.1:37469_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 18588 INFO (qtp2120345233-37) [n:127.0.0.1:37469_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'MoveReplicaHDFSTest_coll_true_shard1_replica_n2' using configuration from collection MoveReplicaHDFSTest_coll_true, trusted=true
[junit4] 2> 18591 INFO (qtp1578008132-34) [n:127.0.0.1:36687_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 18591 INFO (qtp692735685-52) [n:127.0.0.1:38227_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 18593 INFO (qtp692735685-52) [n:127.0.0.1:38227_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.c.CoreContainer Creating SolrCore 'MoveReplicaHDFSTest_coll_true_shard2_replica_n4' using configuration from collection MoveReplicaHDFSTest_coll_true, trusted=true
[junit4] 2> 18591 INFO (qtp1578008132-34) [n:127.0.0.1:36687_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'MoveReplicaHDFSTest_coll_true_shard1_replica_n1' using configuration from collection MoveReplicaHDFSTest_coll_true, trusted=true
[junit4] 2> 18600 INFO (qtp731528774-58) [n:127.0.0.1:37852_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 18600 INFO (qtp731528774-58) [n:127.0.0.1:37852_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.c.CoreContainer Creating SolrCore 'MoveReplicaHDFSTest_coll_true_shard2_replica_n6' using configuration from collection MoveReplicaHDFSTest_coll_true, trusted=true
[junit4] 2> 18601 INFO (qtp731528774-58) [n:127.0.0.1:37852_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37852.solr.core.MoveReplicaHDFSTest_coll_true.shard2.replica_n6' (registry 'solr.core.MoveReplicaHDFSTest_coll_true.shard2.replica_n6') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58ce73ae
[junit4] 2> 18602 INFO (qtp692735685-52) [n:127.0.0.1:38227_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38227.solr.core.MoveReplicaHDFSTest_coll_true.shard2.replica_n4' (registry 'solr.core.MoveReplicaHDFSTest_coll_true.shard2.replica_n4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58ce73ae
[junit4] 2> 18602 INFO (qtp1578008132-34) [n:127.0.0.1:36687_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36687.solr.core.MoveReplicaHDFSTest_coll_true.shard1.replica_n1' (registry 'solr.core.MoveReplicaHDFSTest_coll_true.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58ce73ae
[junit4] 2> 18616 INFO (qtp2120345233-37) [n:127.0.0.1:37469_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37469.solr.core.MoveReplicaHDFSTest_coll_true.shard1.replica_n2' (registry 'solr.core.MoveReplicaHDFSTest_coll_true.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58ce73ae
[junit4] 2> 18630 INFO (qtp1578008132-34) [n:127.0.0.1:36687_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:53179/data
[junit4] 2> 18630 INFO (qtp1578008132-34) [n:127.0.0.1:36687_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 18630 INFO (qtp1578008132-34) [n:127.0.0.1:36687_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 18829 INFO (qtp2120345233-37) [n:127.0.0.1:37469_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:53179/data
[junit4] 2> 18838 INFO (qtp2120345233-37) [n:127.0.0.1:37469_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 18829 INFO (qtp692735685-52) [n:127.0.0.1:38227_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:53179/data
[junit4] 2> 18838 INFO (qtp692735685-52) [n:127.0.0.1:38227_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 18829 INFO (qtp731528774-58) [n:127.0.0.1:37852_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:53179/data
[junit4] 2> 18839 INFO (qtp731528774-58) [n:127.0.0.1:37852_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 18838 INFO (qtp2120345233-37) [n:127.0.0.1:37469_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 18838 INFO (qtp692735685-52) [n:127.0.0.1:38227_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 18839 INFO (qtp731528774-58) [n:127.0.0.1:37852_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 18843 INFO (qtp1578008132-34) [n:127.0.0.1:36687_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.c.SolrCore [[MoveReplicaHDFSTest_coll_true_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_DEE56711D71C82D8-001/tempDir-001/node3/MoveReplicaHDFSTest_coll_true_shard1_replica_n1], dataDir=[hdfs://localhost:53179/data/MoveReplicaHDFSTest_coll_true/core_node3/data/]
[junit4] 2> 18846 INFO (qtp692735685-52) [n:127.0.0.1:38227_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.c.SolrCore [[MoveReplicaHDFSTest_coll_true_shard2_replica_n4] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_DEE56711D71C82D8-001/tempDir-001/node2/MoveReplicaHDFSTest_coll_true_shard2_replica_n4], dataDir=[hdfs://localhost:53179/data/MoveReplicaHDFSTest_coll_true/core_node7/data/]
[junit4] 2> 18846 INFO (qtp731528774-58) [n:127.0.0.1:37852_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.c.SolrCore [[MoveReplicaHDFSTest_coll_true_shard2_replica_n6] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_DEE56711D71C82D8-001/tempDir-001/node1/MoveReplicaHDFSTest_coll_true_shard2_replica_n6], dataDir=[hdfs://localhost:53179/data/MoveReplicaHDFSTest_coll_true/core_node8/data/]
[junit4] 2> 18843 INFO (qtp2120345233-37) [n:127.0.0.1:37469_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.c.SolrCore [[MoveReplicaHDFSTest_coll_true_shard1_replica_n2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_DEE56711D71C82D8-001/tempDir-001/node4/MoveReplicaHDFSTest_coll_true_shard1_replica_n2], dataDir=[hdfs://localhost:53179/data/MoveReplicaHDFSTest_coll_true/core_node5/data/]
[junit4] 2> 18899 INFO (qtp731528774-58) [n:127.0.0.1:37852_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:53179/data/MoveReplicaHDFSTest_coll_true/core_node8/data/snapshot_metadata
[junit4] 2> 18900 INFO (qtp692735685-52) [n:127.0.0.1:38227_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:53179/data/MoveReplicaHDFSTest_coll_true/core_node7/data/snapshot_metadata
[junit4] 2> 18900 INFO (qtp1578008132-34) [n:127.0.0.1:36687_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:53179/data/MoveReplicaHDFSTest_coll_true/core_node3/data/snapshot_metadata
[junit4] 2> 18900 INFO (qtp2120345233-37) [n:127.0.0.1:37469_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:53179/data/MoveReplicaHDFSTest_coll_true/core_node5/data/snapshot_metadata
[junit4] 2> 19067 WARN (qtp2120345233-37) [n:127.0.0.1:37469_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
[junit4] 2> 19068 WARN (qtp692735685-52) [n:127.0.0.1:38227_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
[junit4] 2> 19070 WARN (qtp731528774-58) [n:127.0.0.1:37852_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
[junit4] 2> 19069 WARN (qtp1578008132-34) [n:127.0.0.1:36687_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
[junit4] 2> 24496 INFO (qtp692735685-52) [n:127.0.0.1:38227_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:53179/data/MoveReplicaHDFSTest_coll_true/core_node7/data
[junit4] 2> 24496 INFO (qtp1578008132-34) [n:127.0.0.1:36687_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:53179/data/MoveReplicaHDFSTest_coll_true/core_node3/data
[junit4] 2> 24496 INFO (qtp2120345233-37) [n:127.0.0.1:37469_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:53179/data/MoveReplicaHDFSTest_coll_true/core_node5/data
[junit4] 2> 24514 INFO (qtp731528774-58) [n:127.0.0.1:37852_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:53179/data/MoveReplicaHDFSTest_coll_true/core_node8/data
[junit4] 2> 24573 INFO (qtp731528774-58) [n:127.0.0.1:37852_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:53179/data/MoveReplicaHDFSTest_coll_true/core_node8/data/index
[junit4] 2> 24574 INFO (qtp692735685-52) [n:127.0.0.1:38227_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:53179/data/MoveReplicaHDFSTest_coll_true/core_node7/data/index
[junit4] 2> 24579 INFO (qtp1578008132-34) [n:127.0.0.1:36687_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:53179/data/MoveReplicaHDFSTest_coll_true/core_node3/data/index
[junit4] 2> 24610 INFO (qtp2120345233-37) [n:127.0.0.1:37469_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:53179/data/MoveReplicaHDFSTest_coll_true/core_node5/data/index
[junit4] 2> 25291 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:43829 is added to blk_1073741826_1002{UCState=COMMITTED, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-1b15baba-f70c-47bf-a4ef-819971e60443:NORMAL:127.0.0.1:52789|RBW], ReplicaUC[[DISK]DS-1946115b-85d9-4d58-9e2c-90a734f25961:NORMAL:127.0.0.1:43829|RBW]]} size 69
[junit4] 2> 25291 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:52789 is added to blk_1073741826_1002 size 69
[junit4] 2> 25292 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:43829 is added to blk_1073741827_1003{UCState=COMMITTED, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-bb38408c-b6aa-4f9c-913c-a617542ac63e:NORMAL:127.0.0.1:43829|RBW], ReplicaUC[[DISK]DS-1b15baba-f70c-47bf-a4ef-819971e60443:NORMAL:127.0.0.1:52789|RBW]]} size 69
[junit4] 2> 25292 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:43829 is added to blk_1073741828_1004{UCState=COMMITTED, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-7110f661-e4a3-4b6a-a554-e03695532630:NORMAL:127.0.0.1:52789|RBW], ReplicaUC[[DISK]DS-1946115b-85d9-4d58-9e2c-90a734f25961:NORMAL:127.0.0.1:43829|RBW]]} size 69
[junit4] 2> 25295 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:52789 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-bb38408c-b6aa-4f9c-913c-a617542ac63e:NORMAL:127.0.0.1:43829|RBW], ReplicaUC[[DISK]DS-7110f661-e4a3-4b6a-a554-e03695532630:NORMAL:127.0.0.1:52789|RBW]]} size 0
[junit4] 2> 25295 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:52789 is added to blk_1073741828_1004 size 69
[junit4] 2> 25295 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:52789 is added to blk_1073741827_1003 size 69
[junit4] 2> 25306 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:43829 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-bb38408c-b6aa-4f9c-913c-a617542ac63e:NORMAL:127.0.0.1:43829|RBW], ReplicaUC[[DISK]DS-7110f661-e4a3-4b6a-a554-e03695532630:NORMAL:127.0.0.1:52789|RBW]]} size 0
[junit4] 2> 26485 INFO (qtp692735685-52) [n:127.0.0.1:38227_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 26486 INFO (qtp692735685-52) [n:127.0.0.1:38227_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 26486 INFO (qtp692735685-52) [n:127.0.0.1:38227_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 26489 INFO (qtp731528774-58) [n:127.0.0.1:37852_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 26491 INFO (qtp731528774-58) [n:127.0.0.1:37852_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 26491 INFO (qtp731528774-58) [n:127.0.0.1:37852_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 26494 INFO (qtp2120345233-37) [n:127.0.0.1:37469_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 26494 INFO (qtp2120345233-37) [n:127.0.0.1:37469_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 26494 INFO (qtp2120345233-37) [n:127.0.0.1:37469_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 26501 INFO (qtp1578008132-34) [n:127.0.0.1:36687_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 26502 INFO (qtp1578008132-34) [n:127.0.0.1:36687_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 26502 INFO (qtp1578008132-34) [n:127.0.0.1:36687_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 26548 INFO (qtp692735685-52) [n:127.0.0.1:38227_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 26549 INFO (qtp692735685-52) [n:127.0.0.1:38227_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 26551 INFO (qtp731528774-58) [n:127.0.0.1:37852_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 26553 INFO (qtp731528774-58) [n:127.0.0.1:37852_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 26558 INFO (qtp2120345233-37) [n:127.0.0.1:37469_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 26558 INFO (qtp2120345233-37) [n:127.0.0.1:37469_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 26564 INFO (qtp1578008132-34) [n:127.0.0.1:36687_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 26564 INFO (qtp1578008132-34) [n:127.0.0.1:36687_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 26814 INFO (qtp692735685-52) [n:127.0.0.1:38227_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.s.SolrIndexSearcher Opening [Searcher@15f41a62[MoveReplicaHDFSTest_coll_true_shard2_replica_n4] main]
[junit4] 2> 26815 INFO (qtp731528774-58) [n:127.0.0.1:37852_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.s.SolrIndexSearcher Opening [Searcher@66dece91[MoveReplicaHDFSTest_coll_true_shard2_replica_n6] main]
[junit4] 2> 26818 INFO (qtp2120345233-37) [n:127.0.0.1:37469_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@2066ff82[MoveReplicaHDFSTest_coll_true_shard1_replica_n2] main]
[junit4] 2> 26818 INFO (qtp1578008132-34) [n:127.0.0.1:36687_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5e63147f[MoveReplicaHDFSTest_coll_true_shard1_replica_n1] main]
[junit4] 2> 26850 INFO (qtp1578008132-34) [n:127.0.0.1:36687_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 26850 INFO (qtp731528774-58) [n:127.0.0.1:37852_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 26851 INFO (qtp692735685-52) [n:127.0.0.1:38227_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 26854 INFO (qtp2120345233-37) [n:127.0.0.1:37469_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 26877 INFO (qtp692735685-52) [n:127.0.0.1:38227_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 26877 INFO (qtp731528774-58) [n:127.0.0.1:37852_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 26879 INFO (qtp2120345233-37) [n:127.0.0.1:37469_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 26879 INFO (qtp1578008132-34) [n:127.0.0.1:36687_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 26906 INFO (qtp731528774-58) [n:127.0.0.1:37852_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 26908 INFO (qtp692735685-52) [n:127.0.0.1:38227_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 26908 INFO (qtp2120345233-37) [n:127.0.0.1:37469_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 26909 INFO (qtp1578008132-34) [n:127.0.0.1:36687_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 27223 INFO (searcherExecutor-38-thread-1-processing-n:127.0.0.1:37852_solr x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6 s:shard2 c:MoveReplicaHDFSTest_coll_true r:core_node8) [n:127.0.0.1:37852_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.c.SolrCore [MoveReplicaHDFSTest_coll_true_shard2_replica_n6] Registered new searcher Searcher@66dece91[MoveReplicaHDFSTest_coll_true_shard2_replica_n6] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 27224 INFO (qtp731528774-58) [n:127.0.0.1:37852_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1594131976025014272
[junit4] 2> 27226 INFO (searcherExecutor-35-thread-1-processing-n:127.0.0.1:37469_solr x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2 s:shard1 c:MoveReplicaHDFSTest_coll_true r:core_node5) [n:127.0.0.1:37469_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.c.SolrCore [MoveReplicaHDFSTest_coll_true_shard1_replica_n2] Registered new searcher Searcher@2066ff82[MoveReplicaHDFSTest_coll_true_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 27226 INFO (qtp2120345233-37) [n:127.0.0.1:37469_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1594131976027111424
[junit4] 2> 27239 INFO (searcherExecutor-37-thread-1-processing-n:127.0.0.1:36687_solr x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1 s:shard1 c:MoveReplicaHDFSTest_coll_true r:core_node3) [n:127.0.0.1:36687_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.c.SolrCore [MoveReplicaHDFSTest_coll_true_shard1_replica_n1] Registered new searcher Searcher@5e63147f[MoveReplicaHDFSTest_coll_true_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 27239 INFO (qtp1578008132-34) [n:127.0.0.1:36687_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1594131976040742912
[junit4] 2> 27244 INFO (searcherExecutor-36-thread-1-processing-n:127.0.0.1:38227_solr x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4 s:shard2 c:MoveReplicaHDFSTest_coll_true r:core_node7) [n:127.0.0.1:38227_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.c.SolrCore [MoveReplicaHDFSTest_coll_true_shard2_replica_n4] Registered new searcher Searcher@15f41a62[MoveReplicaHDFSTest_coll_true_shard2_replica_n4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 27244 INFO (qtp692735685-52) [n:127.0.0.1:38227_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1594131976045985792
[junit4] 2> 27277 INFO (qtp692735685-52) [n:127.0.0.1:38227_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.c.ZkShardTerms Successful update terms at /collections/MoveReplicaHDFSTest_coll_true/terms/shard2 to Terms{values={core_node7=0}, version=0}
[junit4] 2> 27286 INFO (qtp1578008132-34) [n:127.0.0.1:36687_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update terms at /collections/MoveReplicaHDFSTest_coll_true/terms/shard1 to Terms{values={core_node3=0}, version=0}
[junit4] 2> 27298 INFO (qtp731528774-58) [n:127.0.0.1:37852_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.c.ZkShardTerms Successful update terms at /collections/MoveReplicaHDFSTest_coll_true/terms/shard2 to Terms{values={core_node7=0, core_node8=0}, version=1}
[junit4] 2> 27299 INFO (qtp2120345233-37) [n:127.0.0.1:37469_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update terms at /collections/MoveReplicaHDFSTest_coll_true/terms/shard1 to Terms{values={core_node3=0, core_node5=0}, version=1}
[junit4] 2> 27312 INFO (qtp1578008132-34) [n:127.0.0.1:36687_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9998ms
[junit4] 2> 27335 INFO (qtp692735685-52) [n:127.0.0.1:38227_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 27336 INFO (zkCallback-33-thread-2-processing-n:127.0.0.1:36687_solr) [n:127.0.0.1:36687_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_coll_true/state.json] for collection [MoveReplicaHDFSTest_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 27336 INFO (qtp692735685-52) [n:127.0.0.1:38227_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 27336 INFO (qtp692735685-52) [n:127.0.0.1:38227_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:38227/solr/MoveReplicaHDFSTest_coll_true_shard2_replica_n4/
[junit4] 2> 27336 INFO (zkCallback-33-thread-1-processing-n:127.0.0.1:36687_solr) [n:127.0.0.1:36687_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_coll_true/state.json] for collection [MoveReplicaHDFSTest_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 27336 INFO (zkCallback-29-thread-2-processing-n:127.0.0.1:37852_solr) [n:127.0.0.1:37852_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_coll_true/state.json] for collection [MoveReplicaHDFSTest_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 27337 INFO (zkCallback-29-thread-1-processing-n:127.0.0.1:37852_solr) [n:127.0.0.1:37852_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_coll_true/state.json] for collection [MoveReplicaHDFSTest_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 27337 INFO (zkCallback-35-thread-2-processing-n:127.0.0.1:37469_solr) [n:127.0.0.1:37469_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_coll_true/state.json] for colle
[...truncated too long message...]
2> 351375 ERROR (SUITE-MoveReplicaHDFSTest-seed#[DEE56711D71C82D8]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 351376 INFO (SUITE-MoveReplicaHDFSTest-seed#[DEE56711D71C82D8]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:53882 53882
[junit4] 2> 351475 INFO (Thread-1) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:53882 53882
[junit4] 2> 351476 WARN (Thread-1) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 27 /solr/collections/MoveReplicaHDFSTest_coll_true/terms/shard2
[junit4] 2> 11 /solr/collections/MoveReplicaHDFSTest_failed_coll_true/terms/shard1
[junit4] 2> 11 /solr/configs/conf1
[junit4] 2> 10 /solr/collections/MoveReplicaHDFSTest_coll_true/terms/shard1
[junit4] 2> 9 /solr/clusterprops.json
[junit4] 2> 9 /solr/aliases.json
[junit4] 2> 7 /solr/collections/MoveReplicaHDFSTest_failed_coll_true/terms/shard2
[junit4] 2> 5 /solr/security.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 207 /solr/collections/MoveReplicaHDFSTest_failed_coll_true/state.json
[junit4] 2> 81 /solr/collections/MoveReplicaHDFSTest_coll_true/state.json
[junit4] 2> 9 /solr/clusterstate.json
[junit4] 2> 2 /solr/collections/MoveReplicaHDFSTest_coll_true/leader_elect/shard2/election/73403349492432905-core_node8-n_0000000001
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 14 /solr/collections
[junit4] 2> 12 /solr/live_nodes
[junit4] 2>
[junit4] 2> 351477 WARN (SUITE-MoveReplicaHDFSTest-seed#[DEE56711D71C82D8]-worker) [ ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
[junit4] 2> 351501 INFO (SUITE-MoveReplicaHDFSTest-seed#[DEE56711D71C82D8]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 351602 WARN (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_DEE56711D71C82D8-001/tempDir-002/hdfsBaseDir/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_DEE56711D71C82D8-001/tempDir-002/hdfsBaseDir/data/data4/]] heartbeating to localhost/127.0.0.1:53179) [ ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
[junit4] 2> 351603 WARN (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_DEE56711D71C82D8-001/tempDir-002/hdfsBaseDir/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_DEE56711D71C82D8-001/tempDir-002/hdfsBaseDir/data/data4/]] heartbeating to localhost/127.0.0.1:53179) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-825003216-10.41.0.5-1520282703353 (Datanode Uuid 09f435c9-ab86-4c51-8a1a-2b5c3cfb7db8) service to localhost/127.0.0.1:53179
[junit4] 2> 351606 WARN (SUITE-MoveReplicaHDFSTest-seed#[DEE56711D71C82D8]-worker) [ ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
[junit4] 2> 351619 INFO (SUITE-MoveReplicaHDFSTest-seed#[DEE56711D71C82D8]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 351721 WARN (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_DEE56711D71C82D8-001/tempDir-002/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_DEE56711D71C82D8-001/tempDir-002/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:53179) [ ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
[junit4] 2> 351721 WARN (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_DEE56711D71C82D8-001/tempDir-002/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_DEE56711D71C82D8-001/tempDir-002/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:53179) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-825003216-10.41.0.5-1520282703353 (Datanode Uuid e3955fe5-61c9-4705-b600-88c0e425407d) service to localhost/127.0.0.1:53179
[junit4] 2> 351755 INFO (SUITE-MoveReplicaHDFSTest-seed#[DEE56711D71C82D8]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 351786 INFO (SUITE-MoveReplicaHDFSTest-seed#[DEE56711D71C82D8]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:53882 53882
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_DEE56711D71C82D8-001
[junit4] 2> Mar 05, 2018 8:50:47 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 35 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {id=FST50}, docValues:{}, maxPointsInLeafNode=1719, maxMBSortInHeap=6.442593653124909, sim=RandomSimilarity(queryNorm=false): {}, locale=ar-IQ, timezone=CNT
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=5,free=170122728,total=318767104
[junit4] 2> NOTE: All tests run in this JVM: [FullHLLTest, MoveReplicaHDFSTest]
[junit4] Completed [65/779 (1!)] on J2 in 362.09s, 4 tests, 1 failure, 1 error, 1 skipped <<< FAILURES!
[...truncated 5636 lines...]
[junit4] Suite: org.apache.solr.analytics.legacy.facet.LegacyFieldFacetCloudTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/contrib/solr-analytics/test/J0/temp/solr.analytics.legacy.facet.LegacyFieldFacetCloudTest_306B9C495D2AE712-001/init-core-data-001
[junit4] 2> Mar 05, 2018 9:36:16 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=true): {}, locale=lt, timezone=America/Araguaina
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=263550616,total=342360064
[junit4] 2> NOTE: All tests run in this JVM: [DateFieldsTest, FilterFunctionTest, ValueFacetTest, LegacyRangeFacetCloudTest, LegacyNoFacetTest, AbsoluteValueFunctionTest, GTFunctionTest, EqualFunctionTest, CastingStringValueStreamTest, LegacyFieldFacetCloudTest]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=LegacyFieldFacetCloudTest -Dtests.seed=306B9C495D2AE712 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=lt -Dtests.timezone=America/Araguaina -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J0 | LegacyFieldFacetCloudTest (suite) <<<
[junit4] > Throwable #1: java.lang.AssertionError: Could not find collection:collection1
[junit4] > at __randomizedtesting.SeedInfo.seed([306B9C495D2AE712]:0)
[junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:155)
[junit4] > at org.apache.solr.analytics.legacy.LegacyAbstractAnalyticsCloudTest.setupCollection(LegacyAbstractAnalyticsCloudTest.java:51)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] Completed [75/78 (1!)] on J0 in 37.89s, 0 tests, 1 failure <<< FAILURES!
[...truncated 42783 lines...]
-ecj-javadoc-lint-src:
[mkdir] Created dir: /tmp/ecj1562568472
[ecj-lint] Compiling 1165 source files to /tmp/ecj1562568472
[ecj-lint] invalid Class-Path header in manifest of jar file: /x1/jenkins/.ivy2/cache/org.restlet.jee/org.restlet/jars/org.restlet-2.3.0.jar
[ecj-lint] invalid Class-Path header in manifest of jar file: /x1/jenkins/.ivy2/cache/org.restlet.jee/org.restlet.ext.servlet/jars/org.restlet.ext.servlet-2.3.0.jar
[ecj-lint] ----------
[ecj-lint] 1. ERROR in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/search/facet/FacetModule.java (at line 44)
[ecj-lint] import org.noggit.JSONUtil;
[ecj-lint] ^^^^^^^^^^^^^^^^^^^
[ecj-lint] The import org.noggit.JSONUtil is never used
[ecj-lint] ----------
[ecj-lint] 1 problem (1 error)
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/build.xml:618: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/build.xml:101: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build.xml:682: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/common-build.xml:2082: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/common-build.xml:2121: Compile failed; see the compiler error output for details.
Total time: 92 minutes 17 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any